Data Warehouse or Data Lake and Suitable ETL Tool for a Startup on AWS

0

Hello,

I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their finances.

They have tasked me with organizing the data aspect of this solution. Our supplier has provided us access to an RDS where they store various tables that generate information about customers, etc. We will be allowed to extract data from these tables.

Table size: Initially, there will be a few tables. The largest tables have about a million rows, growing by 10 to 30 thousand rows monthly.

Update frequency: Updates every 2 hours are acceptable. However, we need real-time updates for one specific table.

I have 2 questions about this:

To be cost-efficient while managing the information: What do you think is the best solution for my company that relies on AWS? Options like Redshift serverless, Redshift Standard, S3 + Athena, Snowflake, PostgreSQL (I know this is OLTP row-oriented but maybe it's cheaper), or something else?

Would Glue be the most economical solution for ETL, or would Lambda + EventBridge or another approach be better?

I appreciate any help you can provide

No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions