By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Lifecycle rule not working

0

I have the following problem: my second lifecycle rule does not seem to do anything.

  • It should move objects in the bucket under a specific path to Glacier IR
  • The objects are uploaded as Intelligent-Tiering. They have been uploaded more than 30 days ago
  • I have a lot of objects in the bucket, but it shouldn't take more than 2 weeks to switch storage class (right?)
  • Bucket versioning is disabled so it's not a version issue
  • I have checked the prefix a bazillion times, it is correct

I have tried the following:

  • The first lifecycle rule works perfectly, to delete certain objects after 30 days
  • I have tried after 60 days, 80, 100, 120, 150 and 180 and nothing seems to work at this point
  • I have already seen several people ask questions here, but all are either extremely basic or didn't read carefully. (I hope I didn't make the same mistake)

Am I missing something or is it just not working?

Enter image description here

  • Tom-B
    a year ago

    What is the size of the objects you're trying to transition? I'm assuming they are MegaBytes in size, if photos, or are they smaller? Also, do you have have any other lifecycle rules active on the bucket?

  • Hans
    a year ago

    Correct. Generally around 4MB or less, and in a very rare case it's some ultra HD image of 10-20MB.

Topics
Storage
Tags
Amazon Simple Storage ServiceAmazon S3 GlacierStorage
Language
English
Hans
asked a year ago275 views
1 Answer
0

I suspect what might be happening here is that this rule is overlapping with another lifecycle policy. If that policy is configured to move data to S3 Intelligent Tiering, then the above lifecycle rule is going to be ignored:

Generally, S3 Lifecycle optimizes for cost. For example, if two expiration policies overlap, the shorter expiration policy is honored so that data is not stored for longer than expected. Likewise, if two transition policies overlap, S3 Lifecycle transitions your objects to the lower-cost storage class. In both cases, S3 Lifecycle tries to choose the path that is least expensive for you. An exception to this general rule is with the S3 Intelligent-Tiering storage class. S3 Intelligent-Tiering is favored by S3 Lifecycle over any storage class, aside from the S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive storage classes.

You can find out more about this here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html#lifecycle-config-conceptual-ex5

AWS
Tom-B
answered a year ago
  • Hans
    a year ago

    Unfortunately this is not the case as each rule has a different prefix. The first one uses prefix backups/ while this one uses photos/. Or is it possible they still somehow overlap even with different prefixes?

  • Tom-B
    a year ago

    I'm curious as to why you're moving these objects from Intelligent Tiering to Glacier Instant Retrieval in the first place? If you wait for 90 days, and the objects aren't accessed, they will move to Archive Instant Access Tier, which is the same price as Glacier Instant Retrieval anyway. Also, by life-cycling these objects, you're going to have to pay a transition fee for them going to Glacier Instant Retrieval.

  • Hans
    a year ago

    Interesting, I was not aware it was charged the same price as Glacier IR. Actually, it was purely because initially a large upload was accidentally uploaded into Intelligent Tiering instead of GIR. After some research, the lifecycle seemed the only way to move such a large number of files to a different storage class. So perhaps the whole move is not needed, however, it is still very strange that the lifecycle is just not working.

  • Tom-B
    a year ago

    One thing to keep in mind for Intelligent Tiering is the small Monitoring Fee that is charged, which would make it slightly more expensive than GIR. Based on the documentation, this transition should work, so I'd double check the date the objects were uploaded vs the number of days you specified in the lifecycle rule. In you original post, you said objects were uploaded more than 30 days ago, but that the lowest value tried was 60 days.

  • Hans
    a year ago

    I see, so moving it to GIR would be preferable. The objects I'm trying to move were uploaded December 9, 2022 until Dec 14, 2022. I started at 60 days, then when nothing happened, I tried higher day counts. Now I tried 180, and by my calculation, that was 187 days ago, that means anything from Dec 9 should have been moved by now but it still hasn't. Is there no way to see whether a rule has effect on an object, like the expiry rules have?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content