Hello everyone,

1. Context

We have a Delta Lake where we write our tables in S3 in Delta format, and we use the Glue Catalog for queries in Athena. Tables are created both with and without the use of Glue crawlers. The crawlers create them using the Delta Native Table format.

2. Problems

Our pipelines have been running smoothly for months. However, since September 15, 2023 (last Friday), we have started to receive several errors in Athena queries. In summary, there are two prominent errors:


Regarding the INVALID_GLUE_SCHEMA error, the first occurrence we have recorded was on September 15, 2023, at 16:35:59.552.

INVALID_GLUE_SCHEMA: Delta Lake table schema in Glue does not match the most recent schema of the Delta Lake transaction log. Please ensure that you have the correct schema defined in Glue.

As for the TABLE_REDIRECTION_ERROR error, the first occurrence we have recorded was on September 15, 2023, at 22:46:52.188.

TABLE_REDIRECTION_ERROR: Table 'awsdatacatalog.silver.core_customers' redirected to 'awsdatacatalog$delta-lake-aws.silver.core_customers', but the target table 'awsdatacatalog$delta-lake-aws.silver.core_customers' does not exist

For an error to occur, a simple query like "SELECT * FROM catalog.table LIMIT 10" is sufficient. Sometimes the query is executed successfully, while other times it results in an error. We didn't make any changes on our end during the creation of the Delta tables.

However, we are noticing an increase in the frequency of this type of error. Currently, it is necessary to execute the query about five times for only one of them to work correctly.

3. Attachments

Enter image description here

Can someone help us? Thank you.

gefragt vor 8 Monaten515 Aufrufe
1 Antwort


For this issue is better to open a support ticket for Amazon Athena. They can see internally how the query is run and see why sometimes the queries run and sometimes not.


profile pictureAWS
beantwortet vor 8 Monaten
profile picture
überprüft vor 5 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen