How to Delete all Dynamodb records except last 1 year (dynamodb table does not have a TTL attribute set)

0

Hello , I need to purge all data from dynamodb table except the last 1 year of data . I do not have a TTL attribute set in the table, what is the best approach to proceed? IT will cost a lot if I write a TTL attribute for every record as per my knowledge although expiring items from TTL is free! I have an attribute called "created_on_date" in the table though !

1 回答
2

You can still enable TTL on created_on_date providing that attribute is an epoch time in seconds.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html

If its not and epoch timestamp in seconds then unfortunately you do not have any other method to approach this that will not involve reading all the items in the table and then either adding TTL or directly deleting.

I would consider backfilling a TTL attribute for all items in the table, which you can compute based on created_on_date. This will ensure you will not run into this situation again. Depending on the amount of data in your table you can either run a script on Lambda/EC2/Local etc... to perform this. For large tables which scale beyond several GB's then you may need to use something such as EMR - this is documented in two blogs:

  1. https://aws.amazon.com/blogs/database/backfilling-an-amazon-dynamodb-time-to-live-ttl-attribute-with-amazon-emr
  2. https://aws.amazon.com/blogs/database/part-2-backfilling-an-amazon-dynamodb-time-to-live-attribute-using-amazon-emr/
profile pictureAWS
专家
已回答 2 年前
profile pictureAWS
专家
Chris_G
已审核 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则