UPSERT RDS tables using AWS Glue Studio Visual and alternatively using Notebook
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual? I know Spark Dataframe through JDBS connections only support Insert / Overwrite, have seen multiple re:Post on this topic, but all are couple of years old. are there any updates to this functionality in the recent times?
If not with Glue Visual, using Notebook, I see an article (https://medium.com/@thomaspt748/how-to-upsert-data-into-relational-database-using-spark-7d2d92e05bb9) explaining how to achieve this using broadcast variables and foreachPartition call approach, is there any better way or alternative to this approach?
- Le plus récent
- Le plus de votes
- La plupart des commentaires
Not out of the box (with the exceptions you mention), you normally do that writing into a temporary table and then issuing a MERGE/UPSERT query into the final table using a separate JDBC connection (so you need a custom code node).
Contenus pertinents
- demandé il y a 4 mois
- demandé il y a un an
- demandé il y a 9 mois
AWS OFFICIELA mis à jour il y a un an- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a 3 ans
