I have a data pipeline built in Redshift Serverless, with some final tables being the result. We are also running a web app that I have set up an Aurora Serverless Postgres DB, to run from. The idea is that all app data will reside in Aurora.
Some of the final tables we have in Redshift are required to build the app. I want to get these tables into Aurora. Rather than run a full data pipeline in Aurora, using the same source data & transformations as Redshift, I want to copy the final tables from Redshift to Aurora.
I was hoping to use AWS Glue to do this. I have been able to validate the Redshift connection and use a crawler to get the Data Catalog set up. But I am struggling with the Aurora connection.
- Do I use the Aurora connection type in the Glue console for Serverless or the JDBC connection type?
- It's Postgres version 15.3 - I tried to use a custom JDBC driver using this thread but still no joy
Is this approach even possible or do I need something else?
AWS OFFICIALUpdated 9 months ago
