- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
Have you considered using Amazon EFS instead of EBS if you need shared storage accessible from multiple EC2 instances? Amazon EFS can be mounted on multiple instances and integrated with services like DataSync for bidirectional sync with S3. Use AWS DataSync to configure ongoing, automated sync between an EFS file system and S3. AWS DataSync can sync in both directions based on filters you define.
https://docs.aws.amazon.com/efs/latest/ug/trnsfr-data-using-datasync.html
Mountpoint for S3 can mount an Amazon S3 bucket as a local file system in your EC2 instance. It supports basic file system operations, list and read existing files, and create new ones. It cannot modify existing files or delete directories, and it does not support symbolic links or file locking. Mountpoint is ideal for applications that do not need all of the features of a shared file system and POSIX-style permissions but require Amazon S3's elastic throughput to read and write large S3 datasets. Refer to Working with Mountpoint for Amazon S3 for details
If you need a POSIX-style storage, consider EFS. Refer to How can I mount an Amazon EFS volume to an instance in my Elastic Beanstalk environment? for overview.
Yes Mike, I would use the S3 bucket rather than EFS.
Running aws s3 sync --delete --size-only s3://mydemobucket/wp-content/uploads/ /var/app/current/wp-content/uploads/
every minute using cron is effectively updating the contents of the uploads
directory on the EC2 to match the uploads
prefix in the bucket, and the --delete
flag means delete anything in the EC2 directory that isn't already in the bucket https://docs.aws.amazon.com/cli/latest/userguide/cli-services-s3-commands.html#using-s3-commands-managing-objects-sync
When one of your developers uploads a file to the directory on the EC2, the next time the cron job runs (which will be within a minute) this new file will be deleted because of the --delete
flag.
You may be able to "beat" this by using something like incron to kick-off a new sync job any time a change is made on the EC2 and sync that file to the bucket. But that's not 100% guaranteed to happen every time.
As other answers have already said, there are other AWS services that are probably a better fit for your use case.
Hi steve, Thanks for the prompt response. I did something. Can you please check below commands? Is it work as I expected?
*/3 * * * * aws s3 sync --delete --size-only s3://examplebucket/wp-content/uploads/ /var/app/current/wp-content/uploads/ */1 * * * * aws s3 sync --size-only /var/app/current/wp-content/uploads/ s3://examplebucket/wp-content/uploads/
I don't know what you're expecting it to do.
I see what it's trying to do - every minute it is sync-ing the EC2
uploads
directory to theuploads
folder in the bucket (and if it finds an object in S3 that is not on EC2 then leave it there). And every three minutes another job syncs the bucket with the directory on EC2 (this time, if a file exists on EC2 that is not in S3 then delete it).This will more-or-less close the gap, but there are still going to be edge cases like when your developer writes a file to the local directory at (for instance) 12:03:00.5 (half a second past 12:03). At this time the first job (EC2 to S3) may have already completed before the file was uploaded. The second job (S3 to EC2) may still be running, and the new file will be found by that and will be deleted.
You could use an event-based daemon like incrond to handle a new file appearing in the local directory, and something using S3 Events to handle a new object in the bucket (e.g. an SQS queue as the destination, and have the EC2 listening for new messages on the queue, process the message and sync the new object as it appears).
It might be worth revisiting the need to run
aws s3 sync
with the--delete
flag in the first place.
Relevanter Inhalt
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 3 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
Yes, We can use If I need to share the files between multiple instances. For my cases, im too using Beanstallk with ASG. However, usually i have to go with only one instance rather than multiple EC2 instances. There is no too much visitors to give much loads.