1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
- I doubt you can make it work correctly, s3 allows that but for the filesystem it means you have a directory without name, I would move the files and avoid issues in the future (even if you can solve it now).
- You need to specify "glue_catalog" so it knows it's the Iceberg catalog, otherwise it will treat it as a regular table.
Contenus pertinents
- demandé il y a 2 mois
- demandé il y a un an
- demandé il y a 4 mois
- Réponse acceptéedemandé il y a 7 mois
- Comment puis-je résoudre les erreurs de connexion à AWS Marketplace dans mes tâches ETL d’AWS Glue ?AWS OFFICIELA mis à jour il y a 3 mois
- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a 2 ans
Thank you Gonzalo for explanation.
One more question, I am running query below from Glue:
I inserted data, I can query it everything seems fine, but when running "SHOW CREATE TABLE {std_database}.{std_table}" on Athena, I am getting error: CREATE TABLE statement cannot be generated because table has unsupported properties.
Both properties I added are described in: https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-creating-tables.html What might be wrong?
Maybe instead of "using" use the table property 'table_type' ='ICEBERG', otherwise it works for me
Hi Honzalo,
I tested various scenarios, and the only way "SHOW CREATE TABLE" on Athena works, when I create table on PySpark without any TBLPROPERTIES.
Also "SHOW CREATE TABLE" answer on Athena and PySpark is different:
Athena:
PySpark:
I think the problem here is that Glue 4.0 creates Iceberg format=1 and Athena is using Iceberg format=2.
Spark defaults to format-version 1, but it should work with 2