Scheduler to fetch data from On-prem db

We have a requirement to sync the data from the on-prem database to AWS RDS (PotgreSQL) at specific intervals (unlike one-time data migration). Assume there is an Interconnect/VPN already established between AWS and On-prem network. The expected data volume is likely 1000 rows only, so I do not see the necessity to build ETL with AWS Glue. Given that, what are the possible solution options to fetch the data?
Can AWS Batch/a pg_cron job be considered here to execute a set of select and update SQLs? Alternatively, how do we trigger the AWS Lambda at certain intervals if AWS Lambda is a solution option for this requirement?

Appreciate your input.

주제

마이그레이션 및 현대화 서버리스 컴퓨팅 데이터베이스 분석

태그

AWS DataSync AWS Lambda PostgreSQL AWS Glue AWS Data Exchange

언어

English

cloudarch

질문됨 일 년 전259회 조회

1개 답변

최신
최다 투표
가장 많은 댓글

이 답변이 도움이 되었나요?커뮤니티가 여러분의 지식을 활용할 수 있도록 정답을 찬성하세요.

Based on the volume, Lambda might be a very good option. The best way to invoke it on a schedule is to use EventBridge Scheduler.

전문가

Uri

답변함 일 년 전

cloudarch
일 년 전
Thanks Uri. But, though EventBridge is a serverless service, do you think an EventBridge event bus is required for this simple daily job scenario? Can't we manage an AWS Batch/a pg_cron job (within PostgreSQL db) to trigger a Lambda at a specific interval?

Scheduler to fetch data from On-prem db

관련 콘텐츠