1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
0
It really depends on how your data is structured. If it's 1 GB file, then it's going to not benefit from Glue being able to fan out. If it's 1024 1MB files, then you're going to see the benefits. Also, it will depend on the block size of the Parquet to allow for optimal I/O (See tip #5 here https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/).
I could only find some information on how to tune your DPUs optimally. The example given was 428 Gzipped JSON files converting to parquet files.
https://docs.aws.amazon.com/glue/latest/dg/monitor-debug-capacity.html
con risposta 5 anni fa
Contenuto pertinente
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata 2 anni fa