How to Delete all Dynamodb records except last 1 year (dynamodb table does not have a TTL attribute set)

0

Hello , I need to purge all data from dynamodb table except the last 1 year of data . I do not have a TTL attribute set in the table, what is the best approach to proceed? IT will cost a lot if I write a TTL attribute for every record as per my knowledge although expiring items from TTL is free! I have an attribute called "created_on_date" in the table though !

1 Respuesta
2

You can still enable TTL on created_on_date providing that attribute is an epoch time in seconds.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/howitworks-ttl.html

If its not and epoch timestamp in seconds then unfortunately you do not have any other method to approach this that will not involve reading all the items in the table and then either adding TTL or directly deleting.

I would consider backfilling a TTL attribute for all items in the table, which you can compute based on created_on_date. This will ensure you will not run into this situation again. Depending on the amount of data in your table you can either run a script on Lambda/EC2/Local etc... to perform this. For large tables which scale beyond several GB's then you may need to use something such as EMR - this is documented in two blogs:

  1. https://aws.amazon.com/blogs/database/backfilling-an-amazon-dynamodb-time-to-live-ttl-attribute-with-amazon-emr
  2. https://aws.amazon.com/blogs/database/part-2-backfilling-an-amazon-dynamodb-time-to-live-attribute-using-amazon-emr/
profile pictureAWS
EXPERTO
respondido hace 2 años
profile pictureAWS
EXPERTO
Chris_G
revisado hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas