Does Data quality work with Apache Iceberg tables

Question

I get an error when trying the recommended rules for an iceberg table:
InvalidInputException (status: 400): Invalid table definition: Parameter "table_type" in table stg_l1_stf_lineitemmap is set to "iceberg" but storage location doesn't conform to the format "s3:////". To specify your warehouse directory, pass it in the request through AdditionalOptions={WarehouseDirectory=}

Also get this error when running DQ rule through a glue job python script that works for other tables:
An error occurred while calling o81.getDynamicFrame. Reads and writes using Lake Formation permissions are not supported for iceberg tables.

Answer

Please refer the below documentation, data quality is supported:  
[1] AWS Glue Data Quality - General availability: new features - https://docs.aws.amazon.com/glue/latest/dg/glue-data-quality.html#data-quality-release-notes-ga

The following new features are available with the general availability of AWS Glue Data Quality:
- Support for Apache Iceberg, Apache Hudi and Delta Lake

Does Data quality work with Apache Iceberg tables

相關內容