Athena time travel queries with Apache Hudi tables

0

Hi,

Is there a way to perform time travel with Athena queries on Apache Hudi tables in a way similar to the one described here Implement a cdc based upsert in a data lake using Apache Iceberg and AWS Glue?
Or the only way is through spark.sql library at the moment?
Thanks

asked 2 months ago44 views
2 Answers
1
Accepted Answer

Hello,

I understand that you would want to know if Athena supports time travel queries with Hudi tables. Unfortunately Athena does not currently support time travel queries for Hudi. There is an existing feature request for the same. We may not be able to provide an ETA on when this will be available. I would suggest to keep an eye on our News page/blog for future announcements. https://aws.amazon.com/new/ https://aws.amazon.com/blogs/aws/

Thank you.

AWS
SUPPORT ENGINEER
answered 2 months ago
0

Are you referring to the MERGE INTO statement that is mentioned in the link? If I understood, your question is - Could the steps mentioned in the Glue job be performed in Athena? Answer is that they cannot be performed directly in Athena as they need a connector, Specific Spark configurations to be set before the spark.sql execution of the MERGE INTO statement.

profile picture
AWS
answered 2 months ago
  • Hi @ananthtm,

    Thanks for your answer.

    I think I am asking something slightly different. I am not asking if the steps mentioned in the Glue job can be performed in Athena, I am asking if it is possible to write the Hudi version of the following Athena query (which is allowed on Iceberg tables)?

    SELECT * FROM iceberg_table FOR SYSTEM_TIME AS OF TIMESTAMP '2022-11-29 18:14:00'

    Thanks!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions