DMS Restart removes the delete markers. - how to handle in GLUE

0

Our Transaction application has a hard delete implemented on MySQL DB on record inactivation scenarios . For the lakehouse , The RDS CDC is exported to S3 through DMS . The problem is , if the DMS is restarted for some reason , the Delete marker of the records get erased , causing orphan records and duplicates in the datalake.

What is the best way to handle this scenario ? Should the glue job transforming data from S3 to the datalake handle such data ? Like a source to target comparison and discard records that are present in the Datalake but not in Source ?

1개 답변
0

To answer your question, we require details that are non-public information. Please open a support case with AWS using the following link:

https://support.console.aws.amazon.com/support/home?#/case/create

AWS
지원 엔지니어
답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠