AWS DMS - Cloud formation - YAML - Migration task not moving from Ready to Starting/Load after instance launch

0

HI, I am able to create the DMS instance (Postgress to S3 migration) through the YAML file[full-load-and-cdc]. It is successfully creating Replication instance, Source and Target end points and migration task. But problem is that Replication task is in 'Ready' state post execution of YAML and not moving into next state like 'Starting' or 'Load complete, replication ongoing'. Noticed no errors. It works well if i use start/resume option on the AWS Console manually. Is any element in YAML script is missing to start the process automatically once the task gets created?

Note: TargetTablePrepMode is set to DO_NOTHING.

First migration job is created manually and second one which I'm working is created by Cloud Formation template.

Enter image description here

Enter image description here

What is equivalent YAML element [action shown in UI] to move the migration task into Starting state ?Below one is option in the console Enter image description here

 TableMappings: !Sub |
      {
      "rules": [
          {
              "rule-type": "selection",
              "rule-id": "132271480",
              "rule-name": "132271480",
              "object-locator": {
                  "schema-name": "${TableMappingSchemaNameFirst}",
                  "table-name": "%"
              },
              "rule-action": "include",
              "filters": [],
              "isAutoSegmentationChecked": false
          },
          {
              "rule-type": "selection",
              "rule-id": "132241247",
              "rule-name": "132241247",
              "object-locator": {
                  "schema-name": "${TableMappingSchemaNameSecond}",
                  "table-name": "%"
              },
              "rule-action": "include",
              "filters": [],
              "isAutoSegmentationChecked": false
          }]
        }

  TargetEndpointArn: !Ref TargetDBEndpoint
  ReplicationTaskSettings: !Sub |
      {
      "StreamBufferSettings": {
          "StreamBufferCount": 3,
          "CtrlStreamBufferSizeInMB": 5,
          "StreamBufferSizeInMB": ${TaskStreamBufferSizeInMB}
      },
      "ErrorBehavior": {
          "FailOnNoTablesCaptured": true,
          "ApplyErrorUpdatePolicy": "LOG_ERROR",
          "FailOnTransactionConsistencyBreached": false,
          "RecoverableErrorThrottlingMax": 1800,
          "DataErrorEscalationPolicy": "SUSPEND_TABLE",
          "ApplyErrorEscalationCount": 0,
          "RecoverableErrorStopRetryAfterThrottlingMax": true,
          "RecoverableErrorThrottling": true,
          "ApplyErrorFailOnTruncationDdl": false,
          "DataTruncationErrorPolicy": "LOG_ERROR",
          "ApplyErrorInsertPolicy": "LOG_ERROR",
          "EventErrorPolicy": "IGNORE",
          "ApplyErrorEscalationPolicy": "LOG_ERROR",
          "RecoverableErrorCount": -1,
          "DataErrorEscalationCount": 0,
          "TableErrorEscalationPolicy": "STOP_TASK",
          "RecoverableErrorInterval": 5,
          "ApplyErrorDeletePolicy": "IGNORE_RECORD",
          "TableErrorEscalationCount": 0,
          "FullLoadIgnoreConflicts": true,
          "DataErrorPolicy": "LOG_ERROR",
          "TableErrorPolicy": "SUSPEND_TABLE"
      },
      "TTSettings": {
          "EnableTT": false
      },
      "FullLoadSettings": {
          "CommitRate": 10000,
          "StopTaskCachedChangesApplied": false,
          "StopTaskCachedChangesNotApplied": false,
          "MaxFullLoadSubTasks": 8,
          "TransactionConsistencyTimeout": 600,
          "CreatePkAfterFullLoad": false,
          "TargetTablePrepMode": "DO_NOTHING"
      },
      "Logging": {
          "EnableLogging": true,
          "EnableLogContext": true,
          "LogComponents": [
              {
                  "Id": "SOURCE_UNLOAD",
                  "Severity": "LOGGER_SEVERITY_DEFAULT"
              },
              {
                  "Id": "SOURCE_CAPTURE",
                  "Severity": "LOGGER_SEVERITY_DEFAULT"
              },
              {
                  "Id": "TARGET_LOAD",
                  "Severity": "LOGGER_SEVERITY_DEFAULT"
              },
              {
                  "Id": "TARGET_APPLY",
                  "Severity": "LOGGER_SEVERITY_DEFAULT"
              },
              {
                  "Id": "TASK_MANAGER",
                  "Severity": "LOGGER_SEVERITY_DEFAULT"
              }
          ],
      },

      "TargetMetadata": {
          "ParallelApplyBufferSize": 0,
          "ParallelApplyQueuesPerThread": 0,
          "ParallelApplyThreads": 0,
          "InlineLobMaxSize": 0,
          "ParallelLoadQueuesPerThread": 0,
          "SupportLobs": false,
          "LobChunkSize": 0,
          "TaskRecoveryTableEnabled": false,
          "ParallelLoadThreads": 0,
          "LobMaxSize": 0,
          "BatchApplyEnabled": false,
          "FullLobMode": false,
          "LimitedSizeLobMode": false,
          "LoadMaxFileSize": 0,
          "ParallelLoadBufferSize": 0
      },
      "ControlTablesSettings": {
          "historyTimeslotInMinutes": 5,
          "HistoryTimeslotInMinutes": 5,
          "StatusTableEnabled": false,
          "SuspendedTablesTableEnabled": false,
          "HistoryTableEnabled": false,
          "ControlSchema": "",
          "FullLoadExceptionTableEnabled": false
      },
      "FailTaskWhenCleanTaskResourceFailed": false,
      "ChangeProcessingTuning": {
          "StatementCacheSize": ${TaskStatementCacheSize},
          "CommitTimeout": 1,
          "BatchApplyPreserveTransaction": true,
          "BatchApplyTimeoutMin": 1,
          "BatchSplitSize": 0,
          "BatchApplyTimeoutMax": 30,
          "MinTransactionSize": ${TaskMinTransactionSize},
          "MemoryKeepTime": 60,
          "BatchApplyMemoryLimit": 500,
          "MemoryLimitTotal": ${TaskMemoryLimitTotal} 
      },
      "ChangeProcessingDdlHandlingPolicy": {
          "HandleSourceTableDropped": true,
          "HandleSourceTableTruncated": true,
          "HandleSourceTableAltered": true
      },}
2개 답변
2
수락된 답변

Hi ashok. You are observing expected behavior. You can use CloudFormation to create and update DMS resources (like a task), but CloudFormation cannot be used to start a task. You can use the AWS CLI, console, or API to manually start the task.

AWS
답변함 2달 전
profile picture
전문가
검토됨 2달 전
0

You can use the below AWS CLI command or start the task from the console manually -

aws dms start-replication-task
--replication-task-arn arn:aws:dms:us-east-1:123456789012:task:K55IUCGBASJS5VHZJIINA45FII
--start-replication-task-type full-load-and-cdc

[+] https://docs.aws.amazon.com/cli/latest/reference/dms/start-replication-task.html [+] https://docs.aws.amazon.com/dms/latest/APIReference/API_StartReplicationTask.html

AWS
지원 엔지니어
답변함 2달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠