How to Delete all Dynamodb records except last 1 year (dynamodb table does not have a TTL attribute set)

0

Hello , I need to purge all data from dynamodb table except the last 1 year of data . I do not have a TTL attribute set in the table, what is the best approach to proceed? IT will cost a lot if I write a TTL attribute for every record as per my knowledge although expiring items from TTL is free! I have an attribute called "created_on_date" in the table though !

1 Resposta
2

You can still enable TTL on created_on_date providing that attribute is an epoch time in seconds.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html

If its not and epoch timestamp in seconds then unfortunately you do not have any other method to approach this that will not involve reading all the items in the table and then either adding TTL or directly deleting.

I would consider backfilling a TTL attribute for all items in the table, which you can compute based on created_on_date. This will ensure you will not run into this situation again. Depending on the amount of data in your table you can either run a script on Lambda/EC2/Local etc... to perform this. For large tables which scale beyond several GB's then you may need to use something such as EMR - this is documented in two blogs:

  1. https://aws.amazon.com/blogs/database/backfilling-an-amazon-dynamodb-time-to-live-ttl-attribute-with-amazon-emr
  2. https://aws.amazon.com/blogs/database/part-2-backfilling-an-amazon-dynamodb-time-to-live-attribute-using-amazon-emr/
profile pictureAWS
ESPECIALISTA
respondido há 2 anos
profile pictureAWS
ESPECIALISTA
Chris_G
avaliado há 2 anos

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas