Tracking record-level failure from Databrew validation

Question

Hi,

This is question with Databrew data profile capability. I am looking for record-level tracking per rule validation status. This will provide while rules has failed on a record from the dataset.

At least, I am looking to separate records after data profiling, valid vs invalid records.

Thanks,

Fabrizio@AWS · Answer

Hello,
this functionality is currently not supported, I will provide feedback to the team owning the service.

Today, to solve your issue you could use a implement in a project recipe the same logic in the quality check you are interested in and using a conditional transform associate a value (true/False or 1/0) to a new column isCol1Valid .

when you run the processing job and write the output to your target you could easily find all rows that are not satisfying the specific check.

I am aware this is currently a workaround.

hope it helps

Tracking record-level failure from Databrew validation

Relevanter Inhalt