DMS form MongoDB to DocumentDB - Migration task aborts with "Connection closed by peer" after 10-15 minutes

Question

Hi guys,

we want to use DMS to migrate a MongoDB (on-prem) to DocumentDB (full load + CDC). The MongoDB is a "shared cluster" of which we use a mongos instance endpoint. One of 8 collections called "XXX" has over 300,000 documents with about 250K each. 
DMS runs well so far, but when loading the "XXX" collection the full load is aborted after approx. 10 - 15 minutes.

CloudWatch Logs:
[SOURCE_UNLOAD ]E: Failed reading from cursor for collection XXX: 'Connection closed by peer' [1028101] (mongodb_unload.c:179)
[SOURCE_UNLOAD ]E: Failed while unloading collection 'XXX' [1028101] (mongodb_unload.c:208)
[TASK_MANAGER ]W: Table 'mymongodb'.'Snapshots' (subtask 2 thread 0) is suspended (replicationtask.c:2550)

We have already tried various variants for the "FullLoadSettings" in the migration task, unfortunately without success.
With this setting, the termination is aborted after 15 minutes:
""FullLoadSettings"": {
    ""TargetTablePrepMode"": ""DROP_AND_CREATE"",
    ""CreatePkAfterFullLoad"": false,
    ""StopTaskCachedChangesApplied"": false,
    ""StopTaskCachedChangesNotApplied"": false,
    ""MaxFullLoadSubTasks"": 8,
    ""TransactionConsistencyTimeout"": 600,
    ""CommitRate"": 10000"

Is there any way to control the connection time from DMS (to prevent "Connection closed by peer") or is this an known issue of MongoDB as a source?
Any ideas are welcome!

Answer

Addition: We found this new MongoDB endpoint setting at DMS 3.4.7 release notes
(https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReleaseNotes.html):

FullLoadNoCursorTimeout => "MongoDB endpoint setting FullLoadNoCursorTimeout specifies NoCursorTimeout for the full load cursor. NoCursorTimeout is a MongoDB connection setting that prevents the server from closing the cursor if idle."

Unfortunately adding it to an existing endpoint as an "Endpoint settings" using the AWS console seems not to work. So we tried to set as an "Extra connection attribute" like this: "authMechanism=DEFAULT;authSource=vdstest;authType=PASSWORD;extractDocId=true;FullLoadNoCursorTimeout=true;nestingLevel=NONE;socketTimeoutMS=3600000;"
This has no effect - After about 10 minutes of full loading the above described error occurs: "Connection closed by peer"

So my second question is: How can we set "FullLoadNoCursorTimeout" for an MongoDB endpoint?

DMS form MongoDB to DocumentDB - Migration task aborts with "Connection closed by peer" after 10-15 minutes

관련 콘텐츠