How do I get accurate Amazon EFS file system and object sizes after a restore in AWS Backup?

Short description

After you restore an EFS file system from a backup, Amazon EFS asynchronously meters and loads files. So, the initial reported size might not be the actual size of the restored data. For example, you restore an EFS file system from a 2.4 GB backup, but the new EFS file system's size is only 6 KB. You must wait for the Amazon EFS metrics to update and access the restored data in the correct location.

Resolution

Check the size of your file system

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Amazon EFS meters files every hour. Check the size of your restored file system about 1 hour or later after Amazon EFS restores or creates the file system. To check the current metered size of the file system, use the AWS CLI or mount the file system.

Note: The reported size in the Amazon EFS console doesn't represent a consistent snapshot of the file system at a point in time. Reported size is based on object sizes that existed in the file system at varying times. The metered size eventually matches the actual sizes of stored objects when there are no writes to the file system.

1. To get information about your EFS file system, run the following AWS CLI describe-file-systems command:

   aws efs describe-file-systems --file-system-id your-file-system-id

   Note: Replace your-file-system-id with your file system ID.

2. To get the current metered size of your EFS file system, check the value for "Value" in the "SizeInBytes" field.

Or, use the Amazon EFS console to view the Total size of your file system. The Total size value reflects the metered size of your file system.

Mount the file system

Complete the following steps:

1. Mount the file system.
2. To show how much disk space the mounted file system uses, run the following df -h command:

   df -h /mnt/efs

The following output shows that the file system is mounted on /home/ec2-user/EFS with a total size of 8.0E, and used space of 5.0G:

Filesystem        Size  Used Avail Use% Mounted on
devtmpfs          4.0M     0  4.0M   0% /dev
tmpfs             453M     0  453M   0% /dev/shm
tmpfs             181M  484K  181M   1% /run
/dev/nvme0n1p1     30G  1.8G   29G   6% /
tmpfs             453M     0  453M   0% /tmp
127.0.0.1:/       8.0E  5.0G  8.0E   1% /home/ec2-user/EFS

Note: The used space is the actual file system size.

Track the disk usage for your files

To track the amount of disk space that your files and directories use, use tools such as du and stat.

To calculate the total size of all files in an EFS mount directory and show the results in gigabytes, run the following command:

find /path/to/efs/mount-directory -type f -ls | awk '{total += $7} END {printf "Total size: %.2f GB\n", total/1024/1024/1024}'

Amazon EFS includes the data size when it uses the space_used attribute to report the space that's used for an object. However, Amazon EFS doesn't include the 2 KiB metadata size. So, a stat command might show a file's size as a multiple of 4 KiB, even when its logical size is smaller.

Monitor file system throughput

Amazon EFS uses different models to report file system throughput depending on the throughput mode that you configured. To view throughput metrics, see Accessing Amazon CloudWatch metrics for Amazon EFS.

View the following throughput metrics:

• For Bursting throughput, monitor BurstCreditBalance.
• For Provisioned throughput, monitor PermittedThroughput.
• To monitor Elastic throughput usage in Amazon EFS, use CloudWatch metrics to track both data operations and metadata operations. For example, to understand your total metadata operation volume, view the MetadataIOBytes metric with the Sum aggregation.

For more information, see Monitoring throughput performance.

Related information

Restore an Amazon EFS file system

Metered size of an EFS file system

Managing mount targets