AWS DMS task is failing on source endpoint timeout

Question

We are running DMS task which is ingesting from the on-prem mssql table with over 200mil records and has LOBs in it. Ingestion takes ~ 33 hours
On-prem DB has a backup that happens after midnight and the task is failing right after that. Below are the logs:

```
[SOURCE_UNLOAD ]D: Network error encountered
[SOURCE_UNLOAD ]D: Is connection error? TRUE
[SOURCE_UNLOAD ]D: Endpoint is disconnected [1020414] (endpointshell.c:3807)
[SOURCE_UNLOAD   ]W:  Error was encountered while FETCH-ing data from table ...
[SOURCE_UNLOAD ]D: Error executing source loop [1020414]
```
I haven't tried the executeTimeout in the source endpoint connection attributes yet and wanted to ask what would be the suggested option to get around the timeout on the source issue

Answer

mostly this could be due to connections from dms are killed or timed out due to backup job. by itself backup command shouldnt disconnect the connection. it basically freezes other db connections for some time.I would suggest using executeTimeout on source endpoint connection attribute in seconds with a value greater than backup time. Being said that do set the remote query timeout to similar value by referring to https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/configure-the-remote-query-timeout-server-configuration-option?view=sql-server-ver15#:~:text=Using%20SQL%20Server%20Management%20Studio,-To%20configure%20the&text=In%20Object%20Explorer%2C%20right%2Dclick,to%20wait%20before%20timing%20out.

AWS DMS task is failing on source endpoint timeout

관련 콘텐츠