- Newest
- Most votes
- Most comments
Hi Tanish,
The maxfilesize
and parallel off
parameters are the two options you have when it comes to limiting the number of files. As you mentioned, parallel off
will increase runtime of the unload, so it is likely not a desirable solution.
I would suggest tweaking the maxfilesize
parameter to get the best output. Keep in mind that each slice in your cluster will create it's own files, so you will be creating at least one file per slice in your cluster, which may be what is introducing the empty files in your unload. Are empty files causing issues with the QuickSight component?
Note you can also change the compression of the output files to further reduce the number of files.
A final point is that QuickSight has the ability to connect directly to Redshift, so if possible in your organisation this may provide better performance and remove the file number problem.
Relevant content
- Accepted Answerasked 15 days ago
- Accepted Answerasked 7 months ago
- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago