一个Data Pipeline中的多个表？

Question

【以下的问题经过翻译处理】 我正在使用Data Pipeline将来自RDS的数据复制到Redshift，对于每个表，都有一个单独的Pipeline进行复制。

是否可能将这些Pipelines合并为一个单独的Pipeline，用于多个表？如果可以，是否有一个示例JSON配置，可以实现这一点？

Answer

【以下的回答经过翻译处理】 是的，这是可能的。下面我包括了一个管道的演示JSON代码。

请注意，由于需要更多的MySql和Redshift属性，因此JSON不完整 - 我还包括了完整属性要求的链接。

MySql DataNode [要求]（https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-mysqldatanode.html）

Redshift DataNode [要求]（https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-redshiftdatanode.html）

```
{
  "objects": [
    {
      "output": {
        "ref": "RedshiftDataNodeId_BWynX"
      },
      "input": {
        "ref": "MySqlDataNodeId_iOnTY"
      },
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultCopyActivity2",
      "id": "CopyActivityId_WMYod",
      "type": "CopyActivity"
    },
    {
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultMySqlDataNode3",
      "id": "MySqlDataNodeId_9uoCY",
      "type": "MySqlDataNode",
      "table": ""
    },
    {
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultRedshiftDataNode1",
      "id": "RedshiftDataNodeId_mJUEj",
      "type": "RedshiftDataNode",
      "tableName": ""
    },
    {
      "output": {
        "ref": "RedshiftDataNodeId_yz06G"
      },
      "input": {
        "ref": "MySqlDataNodeId_9uoCY"
      },
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultCopyActivity3",
      "id": "CopyActivityId_nm5LG",
      "type": "CopyActivity"
    },
    {
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultMySqlDataNode1",
      "id": "MySqlDataNodeId_6Xe5h",
      "type": "MySqlDataNode",
      "table": ""
    },
    {
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultRedshiftDataNode3",
      "id": "RedshiftDataNodeId_yz06G",
      "type": "RedshiftDataNode",
      "tableName": ""
    },
    {
      "failureAndRerunMode": "CASCADE",
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "resourceRole": "DataPipelineDefaultResourceRole",
      "role": "DataPipelineDefaultRole",
      "pipelineLogUri": "s3://athena-ab3/",
      "scheduleType": "cron",
      "name": "Default",
      "id": "Default"
    },
    {
      "occurrences": "1",
      "period": "1 days",
      "name": "RunOnce",
      "id": "DefaultSchedule",
      "type": "Schedule",
      "startAt": "FIRST_ACTIVATION_DATE_TIME"
    },
    {
      "output": {
        "ref": "RedshiftDataNodeId_mJUEj"
      },
      "input": {
        "ref": "MySqlDataNodeId_6Xe5h"
      },
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultCopyActivity1",
      "id": "CopyActivityId_G0Lps",
      "type": "CopyActivity"
    },
    {
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultRedshiftDataNode2",
      "id": "RedshiftDataNodeId_BWynX",
      "type": "RedshiftDataNode",
      "tableName": ""
    },
    {
      "schedule": {
        "ref": "DefaultSchedule"
      },
      "name": "DefaultMySqlDataNode2",
      "id": "MySqlDataNodeId_iOnTY",
      "type": "MySqlDataNode",
      "table": ""
    }
  ],
  "parameters": []
}
```

一个Data Pipeline中的多个表？

相关内容