UPSERT RDS tables using AWS Glue Studio Visual and alternatively using Notebook
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual? I know Spark Dataframe through JDBS connections only support Insert / Overwrite, have seen multiple re:Post on this topic, but all are couple of years old. are there any updates to this functionality in the recent times?
If not with Glue Visual, using Notebook, I see an article (https://medium.com/@thomaspt748/how-to-upsert-data-into-relational-database-using-spark-7d2d92e05bb9) explaining how to achieve this using broadcast variables and foreachPartition call approach, is there any better way or alternative to this approach?
- 最新
- 最多得票
- 最多評論
Not out of the box (with the exceptions you mention), you normally do that writing into a temporary table and then issuing a MERGE/UPSERT query into the final table using a separate JDBC connection (so you need a custom code node).
相關內容
AWS 官方已更新 1 年前- AWS 官方已更新 2 年前
- AWS 官方已更新 3 年前
- AWS 官方已更新 2 年前