How do I set up detailed debug logging for my AWS DMS task?

Resolution

Important: Detailed debug logging requires a large amount of storage space. If you use ongoing change data capture (CDC) replication, then log entries might consume all available storage on the replication instance. Turn off detailed debug logging after you resolve the issue.

By default, DMS uses LOGGER_SEVERITY_DEFAULT for all log components. To troubleshoot migration issues, change the severity to LOGGER_SEVERITY_DETAILED_DEBUG for the specific components that you want to investigate.

Turn on detailed debug logging with the console wizard

The console wizard provides access to the Source Unload, Target Load, Source Capture, Target Apply, and Task Manager standard loggers

To turn on debug logging for any other log component, use the JSON editor or AWS CLI.

Complete the following steps:

1. Open the AWS DMS console.
2. In the navigation pane, choose Tasks.
3. Select your task, and then choose Modify.
4. Under Task Setting, for Editing mode, select Wizard.
5. Select Turn on CloudWatch logs.
6. For each logger that you want to troubleshoot, change the Severity to Detailed debug.
7. Choose Save.

The task generates detailed debug logs on the next run.

Turn on detailed debug logging with the JSON editor

Use the JSON editor to activate debug logging for any DMS log component, including components that aren't available in the console wizard.

Complete the following steps:

Open the AWS DMS console, and then choose Tasks in the navigation pane.

1. Select your task, and then choose Modify.

2. From the Overview details section, expand the Task Settings (JSON) section.

3. For editing mode, select JSON editor.

4. Scroll to the Logging section in the JSON.

5. Change the Severity setting of each component that you want to troubleshoot.
   Example of logging section with detailed debug logging activated for the SOURCE_CAPTURE, METADATA_MANAGER, and COMMON components:

   {
   	"Logging": {
   		"EnableLogging": true,
   		"LogComponents": [{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "TRANSFORMATION"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEBUG",
   				"Id": "SOURCE_UNLOAD"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "IO"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "TARGET_LOAD"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "PERFORMANCE"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DETAILED_DEBUG",
   				"Id": "SOURCE_CAPTURE"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "SORTER"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "REST_SERVER"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "VALIDATOR_EXT"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_INFO",
   				"Id": "TARGET_APPLY"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEBUG",
   				"Id": "TASK_MANAGER"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "TABLES_MANAGER"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DETAILED_DEBUG",
   				"Id": "METADATA_MANAGER"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "FILE_FACTORY"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DETAILED_DEBUG ",
   				"Id": "COMMON"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "ADDONS"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "DATA_STRUCTURE"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "COMMUNICATION"
   			},
   			{
   				"Severity": "LOGGER_SEVERITY_DEFAULT",
   				"Id": "FILE_TRANSFER"
   			}
   		],
   		"CloudWatchLogGroup": null,
   		"CloudWatchLogStream": null
   	}
   }

   Note: Replace LOGGER_SEVERITY_DEFAULT with LOGGER_SEVERITY_DETAILED_DEBUG.

6. Choose Save.

Turn on detailed debug logging with the AWS CLI

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

To modify the task, the task must have a status of Stopped or Failed.

Prerequisite: Install and configure the latest version of the AWS CLI.

Complete the following steps:

1. Open the AWS DMS console, and then choose Tasks in the navigation pane.

2. Choose your task. Then, choose the Details tab.

3. Copy the Task Settings (JSON) and save it locally as a .json file.

4. Turn on CloudWatch logging.

5. Run the following modify-replication-task command based on your operating system (OS):
   Linux:

   aws dms modify-replication-task --replication-task-arn arn:aws:dms:us-east-1:123456789012:task:2PVREMWNPGYJCVU2IBPTOYTIV4 --replication-task-settings file://your-task-settings.json

   Windows:

   aws dms modify-replication-task --replication-task-arn arn:aws:dms:us-east-1:123456789012:task:2PVREMWNPGYJCVU2IBPTOYTIV4 --replication-task-settings file://"c:\temp\your-task-settings.json"

   Note: Replace 123456789012 with the ARN of your replication task and your-task-settings.json with the path to your saved JSON file. To find your Task ARN, open the AWS DMS console, and choose Database migration tasks from the navigation pane. The Task ARN appears in the Overview details section. For more information, see Constructing an Amazon Resource Name (ARN) for AWS DMS.

6. Open the AWS DMS console, and then in the navigation pane, choose Database migration tasks.

7. Confirm that the task status changes to Modifying and then returns to Stopped.

8. From the Overview details section for your task, under Task Settings, confirm that the Severity setting values match your updated JSON file.

The task generates detailed debug logs on the next run.

Related information

Viewing and managing AWS DMS task logs

Logging task settings