Can AWS Glue crawlers crawl parquet files to iceberg type table?

Question

We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?

Gonzalo Herreros · Accepted Answer

The crawler doesn't do any conversion on the data, it registers what you have there.    
So if you just have parquet files and not an Iceberg table, the crawler cannot do that. What the crawler can do is register an Iceberg table in S3 you already have.  
Iceberg has utilities to create a table based on existing files, then you can ask the crawler to register it:  
https://iceberg.apache.org/docs/latest/spark-procedures/#table-migration

Andrii S · Answer

Hello. 
Yes. Please look at this https://aws.amazon.com/about-aws/whats-new/2023/07/aws-glue-crawlers-apache-iceberg-tables/.

Regards, Andrii

Can AWS Glue crawlers crawl parquet files to iceberg type table?

Relevant content