Data Quality statement or SQL query for empty columns

Question

Hello,
Is there a DQ rule that can identify empty or null columns? Or is there an sql query that can identify empty columns?

thanks.

Answer

Hello

According to the document below you could use ColumnLength to check if a column is not empty by checking if col length is greater than 0.  Eg:   ColumnLength "Postal_Code" > 0
https://docs.aws.amazon.com/glue/latest/dg/dqdl.html#dqdl-rule-types-ColumnLength

For not null you could use a custom sql as the example below:
CustomSql "select Name from primary where Age  IS NOT NULL"
https://docs.aws.amazon.com/glue/latest/dg/dqdl.html#dqdl-rule-types-CustomSql

Data Quality statement or SQL query for empty columns

Contenuto pertinente