- Le plus récent
- Le plus de votes
- La plupart des commentaires
To give a full answer would require understanding of the workloads that are running, and in particular what volume of data is being transferred in the cron jobs every five minutes, but it could be a case of the workloads being too much for the resources that it is being run on.
You mention lifecycling your data to IA, is this EFS Standard-IA or OneZone-IA? If EFS is running in Standard mode then you can increase throughput by recreating the filesystem in Max I/O mode https://docs.aws.amazon.com/efs/latest/ug/performance.html#performancemodes
Also,what throughput mode is the EFS filesystem running in https://docs.aws.amazon.com/efs/latest/ug/performance.html#throughput-modes
What are the EC2 instance types, and what are the EBS volume types, and is there any performance hit on these, or is it only the EFS filesystem that is showing a problem?
If EC2 or EBS are a bottleneck then consider an EBS-optimised instance type https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html and/or provisioned IOPS volumes https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html
If EC2 and EBS are comfortably within their limits then one other option that comes to mind is splitting the data across several EFS filesystems instead of just one, and thus spreading the load.
Contenus pertinents
- demandé il y a un an
- demandé il y a 2 mois
- demandé il y a un an
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 2 ans
Okay. My situation is like this,
The EFS storage has 26gb in use. 17GB in standard IA and 9gb in frequently access space. Inside EFS I have 6 directories. As mentioned earlier I have cronjob on 5 instances to do a sync for this directories from efs to local drive (rsync command for each directory) at Sametime( cron set with every 5 min). When we do rsync it will do the read operation for Metadata but I don't have any bulk write or update on all files. Maximum 100 files may updated in 30 min. Since it is read operation most of the files won't go to frequently access space once it moved to IA as they aren't checking this via POSIX way.
All rsync jobs finishing with in 2 min on each box so it's not overlapping with next run
In efs monitoring the Throughput of file system is 100% and when checking iops types it is saying Metadata is all time at 100%.
So I am stuck with where to check without increasing resources. As this behavior started recently.
https://repost.aws/knowledge-center/efs-choosing-correct-throughput-mode
Your filesystem is 26GB which equates to 1.3MB/s.
What does your burst credit balance look like?
https://repost.aws/knowledge-center/efs-burst-credits
You mention that this behaviour started recently. The above link also states:
Could it be you have used up your burst credits, and so now you're maxed out by running constantly at 1.3MB/s?