Incremental Copy of RDS Aurora MySql Table to RedShift

Question

Hi!!!

To copy/incrementally copy data from Aurora MySql 5.7 to RedShift, can anyone share implementation steps? i couldnt do it with DataPipeline, Glue, DMS due to Certificate Issues at the Db.

1. Scheduled Jobs that does S3 exports of initial/changed data. This is the least expensive as you can decide what tables to export data from and how often. You can decide to do that every 10 mins or so. AWS Batch can be used with Lambda to accomplish this. - any references to Achieve this?

Any other better optimised ways?

Thanks!!

Answer

The below link should be helpful for your use-case. The Incremental copy of Amazon RDS MySQL table to Amazon Redshift template copies data from an Amazon RDS MySQL table to an Amazon Redshift table by staging data in an Amazon S3 folder.  This template copies changes that are made to the Amazon RDS MySQL table between scheduled intervals, starting from the scheduled start time. 
[https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-template-redshiftrdsincremental.html]

Incremental Copy of RDS Aurora MySql Table to RedShift

관련 콘텐츠