1 Answer
- Newest
- Most votes
- Most comments
0
Hi rePost-User-9456311,
From my understanding, this seems like this is an issue about the parquet file size since when you reduce the size of the data everything works. I would try to perform a multipart upload with a SDK and see if that solves your issue or not.
Here are some links that should help
https://aws.amazon.com/premiumsupport/knowledge-center/s3-large-file-uploads/
https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
answered 2 years ago
Relevant content
- asked a year ago
- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 9 months ago
- AWS OFFICIALUpdated 2 years ago
Hi, Please tell me the code of the writing part and the size of the file you want to upload. Do you reproduce the problem 100%? Or do you reproduce it once in a while?