Best cost optimization for data warehouse

0

What is the solution that will provide cost optimization:

1- RDS, then AWS Glue ETL and Crawler, then Redshift Serverless

2- RDS, then use RDS Zero ETL, then Redshift Serverless"

2 個答案
2

Hi,

Please, refer to https://aws.amazon.com/rds/aurora/zero-etl/

AWS does not charge an additional fee for Aurora zero-ETL integration 
with Amazon Redshift. You pay for existing Aurora and Amazon Redshift 
resources used to create and process the change data generated as part 
of a zero-ETL integration. These resources could include:

Additional I/O and storage used by enabling change data capture
Snapshot export costs for the initial data export to seed your Amazon 
Redshift databases
Additional Amazon Redshift storage for storing replicated data
Additional Amazon Redshift compute for processing data replication
Cross-AZ data transfer costs for moving data from source to target.
Ongoing processing of data changes by zero-ETL integration is offered 
at no additional charge

On the other side AWS Glue ETL jobs have costs, see pricing page for AWS Glue.

But, Glue ETL jobs allow a deeper transformation of the data that allows you to make you data potentially better aligned with the subsequent queries in Redshift.

So, you may have to re-optimize in Redshift (with costs) data that would have been already optimized by Glue ETL. Glue ETL would then save you this additional processing (either done 1 time or at each query)

So, as you can see, it is not all black-and-white, it really depends on the data of your use case, how it is structured initially, how you need to optimize it for Redshift subsequent queries, etc....

If you work on large-scale project with potentially important costs, the recommended way is to try both approaches to measure on facts which one is optimal cost-wise.

Best,

Didier

profile pictureAWS
專家
已回答 3 個月前
profile picture
專家
已審閱 3 個月前
0

Hello. Glue ETL jobs support more complex transformations but have a cost associated. There is no cost for Zero-ETL, but any complex transformations would have to be done within Redshift. So, in brief, the Zero-ETL option is the more cost-effective.

AWS
已回答 3 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南