Why did my AWS DMS task validation fail, or why isn't the validation progressing?

2 minute read
0

I turned on validation for my AWS Database Migration Service (AWS DMS) task, but the validation failed or isn't progressing.

Short description

When you use the validation feature for an AWS DMS task, you can review statistics at the table level and configure data validation settings.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

To view the data validation information, you can use the AWS DMS console, the AWS CLI, or the AWS DMS API.

Use AWS DMS console

Complete the following steps:

  1. Open the AWS DMS console.
  2. In the navigation pane, choose Database migration tasks.
  3. Choose the name of your task.
  4. From the Table statistics section, review the validation state.

Use the AWS CLI

To review the data validation report in JSON format, run the describe-table-statistics command:

aws dms  describe-table-statistics --replication-task-arn arn:aws:dms:us-east-1:45454545454:rep:XXXXXXXXXXXXXXXXXX

Use the AWS DMI API

Run the DescribeTableStatistics operation.

Troubleshoot the validation

If the validation state is Mismatched records, then query the awsdms_validation_failures_v1 table. AWS DMS creates the awsdms_validation_failures_v1 table on the target database:

select * from awsdms_validation_failures_v1 where TASK_NAME = 'XXXXXXXXXXXXXXXXXX';

Note: The TASK_NAME value is the task's external resource ID and the last value in the task's ARN.

After you have the primary key of the failed record, query the source and target endpoints to see what part of the record doesn't match.

If the validation state is Pending records, then configure the ThreadCount parameter for task validation settings to improve the validation performance. For more information, see Data validation task settings.

For other validation states, see Replication task statistics. You can use Amazon CloudWatch metrics to monitor the progress of the validation.

Related information

Troubleshooting

Limitations

AWS OFFICIAL
AWS OFFICIALUpdated 2 months ago
2 Comments

Is this ERROR as expected? I just want to check on the KEY column

\d awsdms_validation_failures_v1 Table "public.awsdms_validation_failures_v1" Column | Type | Collation | Nullable | Default --------------+-----------------------------+-----------+----------+--------- TASK_NAME | character varying(128) | | not null | TABLE_OWNER | character varying(128) | | not null | TABLE_NAME | character varying(128) | | not null | FAILURE_TIME | timestamp without time zone | | not null | KEY_TYPE | character varying(128) | | not null | KEY | character varying(7800) | | not null | FAILURE_TYPE | character varying(128) | | not null | DETAILS | character varying(7800) | | not null |

select KEY from awsdms_validation_failures_v1 ; ERROR: column "key" does not exist LINE 1: select KEY from awsdms_validation_failures_v1 ; ^ ifet1=> select distinct KEY from awsdms_validation_failures_v1 ; ERROR: column "key" does not exist LINE 1: select distinct KEY from awsdms_validation_failures_v1 ; ^

replied 9 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
EXPERT
replied 9 months ago