AWS DMS Source MongoDB and Target RDS Postgres Full load not happening but ongoing replication happens

Question

I created a DMS task:
Source - MongoDB
Target - RDS Postgres

I have **successfully working pipelines with full load + Ongoing replication working.**

But for some **collections** in my source mongo when I create Full load or Full load + Ongoing replication,
**Task shows successfully completed and starts ongoing replication.**
But Full load doesn't happens only schema gets created in postgres with 0 rows, and Ongoing replication starts.
CloudWatch shows no error even after detailed debug.

Accepted Answer

Thank you for connecting with us. I understand that you already have successful DMS full load and cdc tasks running which are replicating changes from MongoDB source to PostgreSQL target but for certain Collections the full load is not happening at all.

I assume all the configuration has been setup correctly at the MongoDB end as per the following documentation:

[+] Using MongoDB as a source for AWS DMS - 
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html

Given you were not able to find any errors/ warning even with the detailed debug on in the task logs, the only other place to look at will be the target PostgreSQL logs for possible hints around the full load run time.

If the DB logs too don't provide any direction, I will request you to please reach AWS Premium Support Team via a support case and with the task ARN details

Answer

I was able to fix this actually by looking at some non-error logs also,
The issue was the source endpoint which I created had different mongo schema specified.
So in my task one of the table was there with same name in different schema as well, while other was not there.
So in one table full load happened somehow from other schema.

So I started to create different source endpoint for different mongoDB collecctions. So now every thing works as expected

AWS DMS Source MongoDB and Target RDS Postgres Full load not happening but ongoing replication happens

Relevanter Inhalt