DMS Serverless CDC task not capturing newly added PostgreSQL columns

Hello.

Is it possible to view details such as table information by enabling the following settings under "Kinesis endpoint settings"?
https://docs.aws.amazon.com/dms/latest/APIReference/API_KinesisSettings.html

{
  "IncludeControlDetails": true,
  "IncludeTableAlterOperations": true,
  "IncludeNullAndEmpty": true
}

When AWS DMS doesn't capture new columns added to a source table during CDC replication, it's typically because schema evolution isn't properly configured. Here's what you need to know:

Schema Change Handling

AWS DMS can miss new columns if schema evolution isn't activated. To resolve this, you need to modify your task's DDL handling policy:

1. Stop your DMS Serverless replication task
2. Modify the task settings using the JSON editor
3. Locate the ChangeProcessingDdlHandlingPolicy section
4. Set the HandleSourceTableAltered parameter to true

This configuration tells DMS to capture ALTER TABLE operations, including new column additions.

Refreshing the Schema

After enabling schema change handling, you'll need to reload the affected table to include the new columns in your Kinesis output. Simply resuming the task after the configuration change won't retroactively add the missing columns to your data stream - a table reload is necessary to pick up the new schema.

Best Practice

The recommended approach is to configure HandleSourceTableAltered to true before starting CDC replication. This allows DMS to automatically handle schema changes as they occur, preventing the issue you're experiencing.

If you haven't configured this setting from the start, you'll need to stop the task, modify the DDL handling policy as described above, resume the task, and then reload the table to ensure the new columns appear in your replicated data.

This approach applies to DMS tasks generally, including DMS Serverless configurations, and doesn't require recreating your entire replication setup.
Sources
Resolve inconsistent data with Amazon S3 as the target for AWS DMS migration | AWS re:Post