Athena time travel queries with Apache Hudi tables

Question

Hi,

Is there a way to perform time travel with Athena queries on Apache Hudi tables in a way similar to the one described here [Implement a cdc based upsert in a data lake using Apache Iceberg and AWS Glue](https://aws.amazon.com/blogs/big-data/implement-a-cdc-based-upsert-in-a-data-lake-using-apache-iceberg-and-aws-glue/)? \
Or the only way is through *spark.sql* library at the moment? \
Thanks

Accepted Answer

Hello,

I understand that you would want to know if Athena supports time travel queries with Hudi tables. Unfortunately Athena does not currently support time travel queries for Hudi. There is an existing feature request for the same. We may not be able to provide an ETA on when this will be available. I would suggest to keep an eye on our News page/blog for future announcements.
https://aws.amazon.com/new/
https://aws.amazon.com/blogs/aws/

Thank you.

Answer

Are you referring to the  MERGE INTO statement that is mentioned in the link? 
If I understood, your question is - Could the steps mentioned in the Glue job be performed in Athena? 
Answer is that they cannot be performed directly in Athena as they need a connector, Specific Spark configurations to be set before the spark.sql execution of the MERGE INTO statement.

Athena time travel queries with Apache Hudi tables

関連するコンテンツ