DMS Serverless CDC task restarted in RUNNING state but read from PostgreSQL without delivering records to Kinesis

Questions 1 & 2:

confirmed_flush_lsn is clearly explained in the official AWS blog "Best practices to handle AWS DMS tasks during PostgreSQL upgrades | AWS Database Blog" as follows:

confirmed_flush_lsn represents the last LSN that the logical slot consumer has confirmed receiving data to PostgreSQL engine. Data corresponding to the transactions committed before this LSN isn’t available anymore.

As you suspected, this has significant implications. If DMS advances the replication slot before confirming delivery to the target, it implies a possibility that records could be lost without ever reaching the target. This suggests a breakdown in the usual assumption—that the slot advances only after confirming the commit to the target—specifically within this internal restart scenario, rather than it being an intentional design choice. Given the importance of this point, I recommend contacting AWS Support directly to obtain a definitive answer.

Question 3: The AWS Knowledge Center article "Verify if AWS DMS migration task is stuck or making progress | AWS re:Post" recommends monitoring the following combination of metrics for the CDC phase:

CDCThroughputRowsSource (Ingestion rate from source) CDCThroughputRowsTarget (Delivery rate to target) CDCLatencySource (Latency between source and replication instance) CDCLatencyTarget (Latency between replication instance and target)

Your plan to set up an alert based on the condition where CDCThroughputRowsSource > 0 and CDCThroughputRowsTarget = 0 persists for a certain period aligns with official guidance and is a sound approach. In addition, I recommend monitoring CDCLatencyTarget (target-side latency) to ensure it does not show abnormal spikes or remain consistently high.

Your conclusion that monitoring the task status (RUNNING) alone is insufficient is correct. You should monitor not only the status but also the consistency of throughput metrics (specifically, the discrepancy between source and target rates).

Question 4:

Given that no StopReplication or StartReplication API calls were recorded in CloudTrail, I believe your speculation is reasonable—specifically, that the event was likely triggered by the managed capacity management processes internal to DMS Serverless (such as scaling up or host replacement). The fact that this coincided with the increase in CapacityUtilization from 16.8% to 27.5% further supports this hypothesis.

Based on the behavior you've described, this appears to be an abnormal state where the DMS Serverless replication's internal components became desynchronized after the automatic restart. Let me address your questions:

1. Is this a known or expected failure mode?

This is not expected behavior. A properly functioning DMS replication should not continue reading from the source while failing to deliver records to the target without surfacing an error. The condition you're describing—where CDCThroughputRowsSource shows activity but CDCThroughputRowsTarget remains at zero for an extended period—indicates a breakdown in the internal pipeline between the source capture and target apply components.

The log message "No records received to load or apply on target, waiting for data upstream" typically appears when there's genuinely no data to process, not when the source capture component is actively reading changes. This suggests the captured changes weren't being properly passed to the target apply component.

2. Should DMS advance the replication slot's confirmed_flush_lsn without delivering records?

No. DMS should only advance the PostgreSQL replication slot's confirmed_flush_lsn after changes have been successfully applied to the target. The replication slot advancement is meant to signal that DMS has durably processed those changes. Advancing the slot without delivering records to the target represents a potential data loss scenario, as those WAL segments could be recycled by PostgreSQL before the changes are actually replicated.

3. What is the recommended way to detect this condition?

Your approach of monitoring the mismatch between CDCThroughputRowsSource and CDCThroughputRowsTarget is sound. I would recommend:

• Creating a CloudWatch alarm that triggers when CDCThroughputRowsSource is consistently greater than zero while CDCThroughputRowsTarget remains at zero for a sustained period (e.g., 15-30 minutes)
• Monitoring CDCLatencySource and CDCLatencyTarget metrics—significant divergence between these values can also indicate a pipeline issue
• Setting up alerts for unusual patterns in CDCIncomingChanges, CDCChangesMemorySource, or CDCChangesDiskSource, as these may accumulate if changes are being captured but not applied

The replication status alone is insufficient for detecting this condition, as you've observed.

4. Can you determine why DMS Serverless terminated and restarted?

DMS Serverless automatically manages compute resources and may relocate or restart replications for various operational reasons, including capacity management, maintenance, or optimization. The graceful termination you observed is consistent with a planned service-initiated restart rather than a failure.

However, the standard replication task logs and CloudTrail typically don't provide detailed insight into the service's internal decision-making for these operations. For this specific incident, I recommend opening a support case with AWS Support. The support team can access additional service-side diagnostics and telemetry that aren't available through standard monitoring tools. They can investigate:

• Why the replication entered this desynchronized state after restart
• Whether there were any underlying service events that contributed to the issue
• Why no error was surfaced despite the clear delivery failure
• Whether this represents a known issue with a specific DMS engine version or configuration

Given the potential data loss implications of advancing the replication slot without delivering records, this warrants investigation by the DMS service team.
Sources
AWS DMS Serverless components - AWS Database Migration Service
Monitoring AWS DMS tasks - AWS Database Migration Service