Athena time travel queries with Apache Hudi tables

0

Hi,

Is there a way to perform time travel with Athena queries on Apache Hudi tables in a way similar to the one described here Implement a cdc based upsert in a data lake using Apache Iceberg and AWS Glue?
Or the only way is through spark.sql library at the moment?
Thanks

gefragt vor einem Jahr760 Aufrufe
2 Antworten
1
Akzeptierte Antwort

Hello,

I understand that you would want to know if Athena supports time travel queries with Hudi tables. Unfortunately Athena does not currently support time travel queries for Hudi. There is an existing feature request for the same. We may not be able to provide an ETA on when this will be available. I would suggest to keep an eye on our News page/blog for future announcements. https://aws.amazon.com/new/ https://aws.amazon.com/blogs/aws/

Thank you.

AWS
SUPPORT-TECHNIKER
beantwortet vor einem Jahr
AWS
EXPERTE
überprüft vor 8 Monaten
  • Perfect, thanks for confirming!

0

Are you referring to the MERGE INTO statement that is mentioned in the link? If I understood, your question is - Could the steps mentioned in the Glue job be performed in Athena? Answer is that they cannot be performed directly in Athena as they need a connector, Specific Spark configurations to be set before the spark.sql execution of the MERGE INTO statement.

profile pictureAWS
beantwortet vor einem Jahr
  • Hi @ananthtm,

    Thanks for your answer.

    I think I am asking something slightly different. I am not asking if the steps mentioned in the Glue job can be performed in Athena, I am asking if it is possible to write the Hudi version of the following Athena query (which is allowed on Iceberg tables)?

    SELECT * FROM iceberg_table FOR SYSTEM_TIME AS OF TIMESTAMP '2022-11-29 18:14:00'

    Thanks!

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen