AWS DMS task is failing on source endpoint timeout

0

We are running DMS task which is ingesting from the on-prem mssql table with over 200mil records and has LOBs in it. Ingestion takes ~ 33 hours On-prem DB has a backup that happens after midnight and the task is failing right after that. Below are the logs:

[SOURCE_UNLOAD ]D: Network error encountered
[SOURCE_UNLOAD ]D: Is connection error? TRUE
[SOURCE_UNLOAD ]D: Endpoint is disconnected [1020414] (endpointshell.c:3807)
[SOURCE_UNLOAD   ]W:  Error was encountered while FETCH-ing data from table ...
[SOURCE_UNLOAD ]D: Error executing source loop [1020414]

I haven't tried the executeTimeout in the source endpoint connection attributes yet and wanted to ask what would be the suggested option to get around the timeout on the source issue

1 Answer
0

mostly this could be due to connections from dms are killed or timed out due to backup job. by itself backup command shouldnt disconnect the connection. it basically freezes other db connections for some time.I would suggest using executeTimeout on source endpoint connection attribute in seconds with a value greater than backup time. Being said that do set the remote query timeout to similar value by referring to https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/configure-the-remote-query-timeout-server-configuration-option?view=sql-server-ver15#:~:text=Using%20SQL%20Server%20Management%20Studio,-To%20configure%20the&text=In%20Object%20Explorer%2C%20right%2Dclick,to%20wait%20before%20timing%20out.

AWS
answered 2 years ago
  • Thank you for the answer. I have added executeTimeout to the source however it hasn't changed the behavior and the task failed again. I have noticed following in the debug logs: [SOURCE_UNLOAD ]D: Setting connection timeout as : '8' (ar_odbc_conn.c:533) Not sure what setting is controlling this parameter. I haven't checked the source DB yet - passed it to the customer.

  • Hi, It's been some time since posted, but did you or your customer find any solution for this?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions