How to Delete all Dynamodb records except last 1 year (dynamodb table does not have a TTL attribute set)

0

Hello , I need to purge all data from dynamodb table except the last 1 year of data . I do not have a TTL attribute set in the table, what is the best approach to proceed? IT will cost a lot if I write a TTL attribute for every record as per my knowledge although expiring items from TTL is free! I have an attribute called "created_on_date" in the table though !

1개 답변
2

You can still enable TTL on created_on_date providing that attribute is an epoch time in seconds.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html

If its not and epoch timestamp in seconds then unfortunately you do not have any other method to approach this that will not involve reading all the items in the table and then either adding TTL or directly deleting.

I would consider backfilling a TTL attribute for all items in the table, which you can compute based on created_on_date. This will ensure you will not run into this situation again. Depending on the amount of data in your table you can either run a script on Lambda/EC2/Local etc... to perform this. For large tables which scale beyond several GB's then you may need to use something such as EMR - this is documented in two blogs:

  1. https://aws.amazon.com/blogs/database/backfilling-an-amazon-dynamodb-time-to-live-ttl-attribute-with-amazon-emr
  2. https://aws.amazon.com/blogs/database/part-2-backfilling-an-amazon-dynamodb-time-to-live-attribute-using-amazon-emr/
profile pictureAWS
전문가
답변함 2년 전
profile pictureAWS
전문가
Chris_G
검토됨 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠