Best cost optimization for data warehouse

0

What is the solution that will provide cost optimization:

1- RDS, then AWS Glue ETL and Crawler, then Redshift Serverless

2- RDS, then use RDS Zero ETL, then Redshift Serverless"

2 Risposte
2

Hi,

Please, refer to https://aws.amazon.com/rds/aurora/zero-etl/

AWS does not charge an additional fee for Aurora zero-ETL integration 
with Amazon Redshift. You pay for existing Aurora and Amazon Redshift 
resources used to create and process the change data generated as part 
of a zero-ETL integration. These resources could include:

Additional I/O and storage used by enabling change data capture
Snapshot export costs for the initial data export to seed your Amazon 
Redshift databases
Additional Amazon Redshift storage for storing replicated data
Additional Amazon Redshift compute for processing data replication
Cross-AZ data transfer costs for moving data from source to target.
Ongoing processing of data changes by zero-ETL integration is offered 
at no additional charge

On the other side AWS Glue ETL jobs have costs, see pricing page for AWS Glue.

But, Glue ETL jobs allow a deeper transformation of the data that allows you to make you data potentially better aligned with the subsequent queries in Redshift.

So, you may have to re-optimize in Redshift (with costs) data that would have been already optimized by Glue ETL. Glue ETL would then save you this additional processing (either done 1 time or at each query)

So, as you can see, it is not all black-and-white, it really depends on the data of your use case, how it is structured initially, how you need to optimize it for Redshift subsequent queries, etc....

If you work on large-scale project with potentially important costs, the recommended way is to try both approaches to measure on facts which one is optimal cost-wise.

Best,

Didier

profile pictureAWS
ESPERTO
con risposta 3 mesi fa
profile picture
ESPERTO
verificato 3 mesi fa
0

Hello. Glue ETL jobs support more complex transformations but have a cost associated. There is no cost for Zero-ETL, but any complex transformations would have to be done within Redshift. So, in brief, the Zero-ETL option is the more cost-effective.

AWS
con risposta 3 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande