Hello everyone,

1. Context

We have a Delta Lake where we write our tables in S3 in Delta format, and we use the Glue Catalog for queries in Athena. Tables are created both with and without the use of Glue crawlers. The crawlers create them using the Delta Native Table format.

2. Problems

Our pipelines have been running smoothly for months. However, since September 15, 2023 (last Friday), we have started to receive several errors in Athena queries. In summary, there are two prominent errors:


Regarding the INVALID_GLUE_SCHEMA error, the first occurrence we have recorded was on September 15, 2023, at 16:35:59.552.

INVALID_GLUE_SCHEMA: Delta Lake table schema in Glue does not match the most recent schema of the Delta Lake transaction log. Please ensure that you have the correct schema defined in Glue.

As for the TABLE_REDIRECTION_ERROR error, the first occurrence we have recorded was on September 15, 2023, at 22:46:52.188.

TABLE_REDIRECTION_ERROR: Table 'awsdatacatalog.silver.core_customers' redirected to 'awsdatacatalog$delta-lake-aws.silver.core_customers', but the target table 'awsdatacatalog$delta-lake-aws.silver.core_customers' does not exist

For an error to occur, a simple query like "SELECT * FROM catalog.table LIMIT 10" is sufficient. Sometimes the query is executed successfully, while other times it results in an error. We didn't make any changes on our end during the creation of the Delta tables.

However, we are noticing an increase in the frequency of this type of error. Currently, it is necessary to execute the query about five times for only one of them to work correctly.

3. Attachments

Enter image description here

Can someone help us? Thank you.

demandé il y a 8 mois572 vues
1 réponse


For this issue is better to open a support ticket for Amazon Athena. They can see internally how the query is run and see why sometimes the queries run and sometimes not.


profile pictureAWS
répondu il y a 8 mois
profile picture
vérifié il y a 5 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions