Inconsistent documentation for DQDL rule type IsComplete

0

I get strange errors in the data quality tab of the glue studio, which seem to contradict the documentation in https://docs.aws.amazon.com/glue/latest/dg/dqdl.html#dqdl-rule-types-IsComplete:

Rule_13 IsComplete "abc"
Rule failed	Unsupported nested column type of column abc: ArrayType(DoubleType,true)!

Contradicts: "Supported column types: Any column type"

Rule_14 IsComplete "xyz"
Rule failed	Value: 0.6140889354693816 does not meet the constraint requirement!

Contradicts: "Checks whether all of the values in a column are complete (non-null)."

Is there any better way to understand what is going wrong in my case?

demandé il y a un an253 vues
1 réponse
0

Hi, thank you for your feedback on the AWS Glue Data Quality preview. As for your specific questions:

  1. in the first page of the documentation on the DataQuality evaluation we mention that :"Data quality rules can't evaluate nested or list-type data sources.". So later we refer to all supported column types, still we will keep into consideration your feedback.
  2. The error message you get is specifying that the rule evaluation failed because 61.4% of the rows are actually null and that violates the constraint for the rule.

hope this helps.

AWS
EXPERT
répondu il y a un an

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions