Does Data quality work with Apache Iceberg tables

0

I get an error when trying the recommended rules for an iceberg table: InvalidInputException (status: 400): Invalid table definition: Parameter "table_type" in table stg_l1_stf_lineitemmap is set to "iceberg" but storage location doesn't conform to the format "s3://<warehouse-dir>/<glue database>/<glue table id>". To specify your warehouse directory, pass it in the request through AdditionalOptions={WarehouseDirectory=<your-warehouse-dir>}

Also get this error when running DQ rule through a glue job python script that works for other tables: An error occurred while calling o81.getDynamicFrame. Reads and writes using Lake Formation permissions are not supported for iceberg tables.

neal_li
demandé il y a 6 mois316 vues
1 réponse
0

Please refer the below documentation, data quality is supported:
[1] AWS Glue Data Quality - General availability: new features - https://docs.aws.amazon.com/glue/latest/dg/glue-data-quality.html#data-quality-release-notes-ga

The following new features are available with the general availability of AWS Glue Data Quality:

  • Support for Apache Iceberg, Apache Hudi and Delta Lake
AWS
répondu il y a 6 mois
  • Then why am i getting an error "An error occurred while calling o81.getDynamicFrame. Reads and writes using Lake Formation permissions are not supported for iceberg tables."

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions