使用 AWS re:Post 即表示您同意 AWS re:Post 使用條款
AWS re:Postre:Post

DMS IS NOT REPLICATING TO S3 THE NEW COLUMNS ADDED TO THE SOURCE DDL

0

I have created a replication task with Postgres as source and S3 as destination and I have noticed that it cannot replicate columns created after the start of the task. After the changes in the source table have been added, I create a task with exactly the same characteristics and in this way there is no problem to do the replication with the new columns Is there a way to capture the new columns without restarting the task? NOTE: The parameter that seems to work with this stays like this "HandleSourceTableAltered": true

**********************UPDATE: It is not enough to restart the task, apparently the only way to bring the new fields is by creating a new task only with the full load and with the CSV format, with Parquet the new columns are not migrated, if I activate the CDC configuration, neither I get the new columns

主題
儲存空間遷移與現代化資料庫開發人員工具
標籤
Amazon Simple Storage ServiceAWS Database Migration ServicePostgreSQL開發人員工具提取轉換與載入資料
語言
English
rePost-User-8799758
已提問 1 年前檢視次數 1426 次
2 個答案
1

When using AWS DMS, keep in mind that there are some limitations when it comes to capturing schema changes. When you add new columns to the source database, AWS DMS may not replicate these new columns automatically. As you have observed, restarting the replication task is usually required to resolve this issue. This is due to the replication task's need to reload the source tables' metadata in order to understand the new schema.

about the parameter yes that parameter can capture DDL events

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.DDLHandling.html

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.html#CHAP_Tasks.CustomizingTasks.TaskSettings.Example

profile picture
專家
Sedat Salman
已回答 1 年前
  • rePost-User-8799758
    1 年前

    Thanks for the answer, it is very helpful So what is the exact function of the HandleSourceTableAltered parameter? I have it set to True but it can't capture the changes in the DDL

  • rePost-User-8799758
    1 年前

    I have noticed that restarting the task is not enough, at the moment the only solution is to create a new task with the same endpoints

  • AQM
    9 個月前

    I have an RDS MySQL source and a Redshift target. I added a column to a source table, but the task failed to capture it, and even after restarting the task many times, it is in Running with Errors state.

    My task settings have this:

     "ChangeProcessingDdlHandlingPolicy": {
        "HandleSourceTableDropped": true,
        "HandleSourceTableTruncated": true,
        "HandleSourceTableAltered": true
    },

    But still DMS is not capturing changes and task is in an error state.

    What's the solution to this? Surely, re-creating the task from start is not a viable option.

0

We had the exact same issue. After configuring

"CaptureDdls": true,

on DMS Postgres endpoint, new columns get captured instantaneously. Hope this works for you as well.

Matthias
已回答 8 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋，同時有助於提問者的專業成長。

回答問題指南

相關內容