AWS Glue: ETL Jobs Connection bug

0

I have found that when multiple connections are used in a Glue ETL job, the order of the connections relative to the data source and data target nodes matter. For example, I have a SQL server data source node and a Redshift target node. When I make any changes the connection order swaps effectively making the first connection in the job details tab my Redshift connection and the next the SQL server connection. The ETL job fails with a cannot connect error when trying to connect to SQL server since it's not the first connection in the job details. The only solution i've found is deleting the target node and reintroducing it to fix the job.

Solomon
質問済み 7ヶ月前198ビュー
2回答
1
承認された回答

That's correct, you can have multiple connections in terms of driver and configuration, but only one network configuration can be used (the job cannot be on multiple VPC/subnets at the same time).
https://docs.aws.amazon.com/glue/latest/dg/glue-troubleshooting-errors.html#vpc-failover-behavior-error-10
For that you would need a single VPC that can reach both systems (e.g. peering)

profile pictureAWS
エキスパート
回答済み 7ヶ月前
profile picture
エキスパート
レビュー済み 2ヶ月前
0

Thank you for clarifying.

Solomon
回答済み 7ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ