- Newest
- Most votes
- Most comments
The low throughput and high latency you're experiencing most likely relates to the lazy loading of blocks from S3 when EBS volumes are restored from snapshots (including AMIs). See explanation of lazy loading below:
For volumes that were created from snapshots, the storage blocks must be pulled down from Amazon S3 and written to the volume before you can access them. This preliminary action takes time and can cause a significant increase in the latency of I/O operations the first time each block is accessed. Volume performance is achieved after all blocks have been downloaded and written to the volume.
Reference: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html
How can you fix this? Review the recommendations as explained in this blog Addressing I/O latency when restoring Amazon EBS volumes from EBS Snapshots
There's pros and cons with both, i.e. EBS initialization is a manual action but free, and FSR instantly delivers all of the EBS provisioned performance but has a cost associated with it. You will have to decide which of the two options is more suitable.
I would suggest trying the following:
- Without FSR, is the gp3 EBS volume close to hitting its IOPS or throughput limit? If so, increase the provision IOPS (max of 16K), and potentially the provisioned throughput (max of 1,000MiB/s), to see if this improve disk read performance on instance startup.
- Add a startup script to the AMI that pre-initializes the volume after the instance launches.
- Since write performance is not affected by lazy loading, try copying the 6GB file onto the instance (e.g from S3) via a startup script.
In terms of if or how FSR works with Marketplace AMIs. There doesn't appear to be a precedent for this. I'd suggest contacting AWS Support in relation to this query.
Relevant content
- asked 5 months ago
- asked 7 months ago
- asked 4 years ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 9 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
AntAWS, I suspected this might be the case, and I tested with FSR and had slightly better performance. (My startup scripts ran ~2x faster, still slow compared to my expectations, but application startup went from 10m to 5m.
HOWEVER, I would like to release my AMIs into the marketplace, and FSR is very expensive and would also then need to be enabled in every region my AMI is. How does this work with Marketplace AMIs?