Skip to content

AWS Data Migration Service (DMS) with RDS Aurora Postgres

0

Is it possible to configure AWS Database Migration Service (DMS) in Full Load + CDC mode using an Amazon Aurora PostgreSQL Read Replica as the source? The goal is to capture ongoing data changes and replicate them to an S3-based data lake, while avoiding any additional load on the primary (writer) instance.

1 Answer
1
Accepted Answer

Based on AWS DMS capabilities with Aurora PostgreSQL, using a Read Replica as the source for Full Load + CDC mode is not a recommended approach. Here's why:

For Change Data Capture (CDC) to work with PostgreSQL databases, AWS DMS requires access to the database's write-ahead logs (WAL) and uses logical replication slots. CDC functionality specifically needs to capture changes as they occur, which requires connecting to the source where these changes are being written and logged.

The proper configuration for Aurora PostgreSQL as a source involves:

  1. Enabling logical replication on the source database by setting parameters like rds.logical_replication, wal_level, max_replication_slots, and max_wal_senders
  2. Ensuring the user account has the necessary roles (rds_superuser and rds_replication for AWS-managed PostgreSQL databases)
  3. Creating the source endpoint pointing to the writer instance where the logical replication slot will be created

While your goal of avoiding additional load on the primary instance is understandable, CDC operations inherently need to connect to where the changes are occurring and being logged. The read replica doesn't maintain its own independent logical replication infrastructure for CDC purposes.

For Full Load + CDC tasks with Aurora PostgreSQL, you should configure AWS DMS to connect to the primary (writer) instance. The CDC process is designed to be efficient and minimally impactful, as it reads from the transaction logs rather than querying tables directly.
Sources
Using a PostgreSQL database as an AWS DMS source - AWS Database Migration Service
Example: Logical replication using Aurora PostgreSQL and AWS Database Migration Service - Amazon Aurora

answered 12 days ago
EXPERT
reviewed 11 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.