Uploading restrictions for large amount of files in S3

Question

Are there any restrictions like max amount of time for an upload of a large amount of files (e.g. 15 million files with no more than 1MB each) ? 
Like that the upload stops after a max number of files uploaded, or after a specific time of the upload running ?

Matt-B · Answer

Hi There
That might depend on what method are you using to upload them.  If you are using the CLI like the `aws s3 sync` command, you could be limited by the shell session timeout, or other transient issues like a short network issue that stops the transfer.  You would need to manually run the sync command again.  You can also use a managed service like [DataSync](https://aws.amazon.com/blogs/storage/migrating-hundreds-of-tb-of-data-to-amazon-s3-with-aws-datasync/) that will handle this for you.

Uploading restrictions for large amount of files in S3

Relevant content