Glue catalog cross account - Athena query
Hi everyone, for Glue catalog cross account I changed the key in catalog settings to a customer-managed kms key. I kept s3 bucket(where data stores) encryption type as it was( SSE-s3). I also did all step we need for cross account. after that, when I create a new table, it is queryable inside consumer and producer account, I can not run query for existing tables in consumer account (seems they are encrypted by old key). I re-trigger their crawlers. but still there is the error there. so how we can re-encrypt whole metadata for those tables with new key?
note: if I create new crawler with the same data, it will work! but existing tables not.
- Newest
- Most votes
- Most comments
The issue you’re facing is likely due to the fact that the existing tables in your Glue catalog were created before the KMS key was changed. These tables are still associated with the old key, and that’s why you’re unable to query them from the consumer account.
Unfortunately, AWS Glue doesn’t provide a built-in way to re-encrypt the metadata of existing tables with a new key. The metadata encryption is determined at the time of table creation and can’t be changed afterwards.
So either you recreate the Tables or use AWS Glue Data Catalog Client
Relevant content
- asked a year ago
AWS OFFICIALUpdated 7 months ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
thanks for answering. what do you mean by 'use AWS Glue Data Catalog Client' ? how it can help?
The AWS Glue Data Catalog client can be used to manage the metadata of your data stored in Amazon S3. It provides a unified metadata repository across various services, enabling you to query your data across data stores.
In the context of your issue, you could use the AWS Glue Data Catalog client to programmatically recreate the tables. This would involve dropping the existing tables and creating new ones, which would then be encrypted with the new key. Here's a high-level overview of how you might do this: