S3 Object Lock and incomplete uploads.


Say you want to upload a bunch of really large files. However somewhere in between the upload something goes wrong. The files don't complete.

The whole thing is a mess so you just want to start over. If you set up the bucket with Object Lock in Compliance mode. You're pretty much just going to have to eat the costs for the durations of the retention period? Am I understanding this correctly?

Do you guys have any advice for dealing with these situations? This is for backups I am currently using Amazon S3 (not Glacier). Perhaps I should be using Amazon S3 Glacier with Vault lock instead. But I'm not as familiar with it as I've never used it before.

Is there a better way to handle this situation? Ultimately I want the files to be immutable after I upload them. But I wouldn't mind if it waited until the upload finished.

(SEPARATE QUESTION) Another problem I run into is my S3 Browser uploads everything as "Standard". When I change everything to Glacier Deep Storage it redownloads the data again. While also creating a separate version still in Standard. Am I going to be charged for both? And if so is there any way to avoid this besides using the funky browser upload? (That is how I got the messed up original upload)

So in summation:

  1. How does Object Lock handle incomplete uploads. Is there anyway to delete them after you uploaded them.
  2. Is there a better way to apply a different storage tier to an Object Locked S3 Bucket. Besides uploading it twice or using the browser.

Thank you!

asked a year ago637 views
1 Answer

This doesn't answer your question, but it might save you some money. If you're doing multipart uploads to S3, you need to know to create a lifecycle rule that deletes failed multipart uploads from each bucket that you upload to. Here's a document on how to do that. I don't think you need to wait 7 days for it to take effect though. You might want to choose a shorter time frame. https://aws.amazon.com/blogs/aws-cloud-financial-management/discovering-and-deleting-incomplete-multipart-uploads-to-lower-amazon-s3-costs/

answered a year ago
  • If you do find you're being charged for multipart failed uploads, you can request a refund from AWS. Just open a case and tell them which buckets you were uploading to and how far back this uploading was being done and they will take care of it.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions