- Mais recentes
- Mais votos
- Mais comentários
Hello,
Firstly, Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. If you are seeing only the data from the latest version in Athen that means you only have the latest version of data in the underlying S3 folder. As per your description it seems that the new versions is overwriting the old version in S3.
Secondly, if you are able to compare versions in Glue that means that a new version of table is getting added to glue catalog each time data is unloaded into S3.
Lastly, if you are looking for an ability to query all the versions of your table snapshot. You have to unload and store the data of each new version as a new folder under products (example version_1, version_2 etc ) and add the same to glue catalog. You can consider using glue crawler [1] for adding the new partitions to the table.
————————
Reference:
[1] https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html
================
Have a nice day!
Hi Arun,
Thanks for answering. I did what you mentioned, (create a crawler to add the new partitions to the table). But the result was just 2 different tables (1 table for each version I guess), and each table starts with a name "part_00000_c5e9..." . Also the link you sent is quite general, could you please specify the documentation that actually aggregates the different partitions/versions into one table?
Thank you
Conteúdo relevante
- AWS OFICIALAtualizada há 2 anos
- AWS OFICIALAtualizada há 2 anos
- AWS OFICIALAtualizada há 2 meses
The version parameter is the bigint snapshot ID associated with a governed table version.