2 個答案
- 最新
- 最多得票
- 最多評論
1
When DMS replicates data from a database to S3 and enables Glue catalog generation, it sets certain properties in the generated Glue tables. One such property is escape.delim, which gets set to null.
This null value does not cause issues when querying the data from Athena. However, it can cause problems when trying to access the tables from Spark using the create_dynamic_frame_from_catalog option, as Spark expects a non-null escape delimiter value.
There is currently no option in DMS to configure this escape.delim property value.
- After the initial load and replication is complete, update the Glue table definition manually through the Glue console or API to set a non-null escape delimiter value.
- Alternatively, instead of using create_dynamic_frame_from_catalog in Spark, you can directly query the data from S3 using Spark SQL without going through the Glue catalog.
0
I noticed that when using create_dynamic_frame_from_options
and reading directly from S3 I don't have the same problem, I was curious as to why that was the case. Thank you! Now it's clear
已回答 1 個月前
相關內容
- AWS 官方已更新 1 年前
- AWS 官方已更新 2 年前