2 Risposte
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
0
This Issue is also filed. After changing the S3 Event type to "All Object Create Events" I was able to process the huge file (230MB) also.
con risposta 7 mesi fa
0
The S3 trigger should invoke the Lambda function regardless of the file size uploaded. Can you confirm in CloudWatch logs that the Lambda is not invoked at all. The more likely scenario is that either the S3 object is not being successfully uploaded, or that the Lambda function is in fact being invoked but experiencing an error during execution. If confident neither of these scenarios apply, please provide some additional context into how the S3 trigger was created.
con risposta 7 mesi fa
Contenuto pertinente
AWS UFFICIALEAggiornata 4 mesi fa- AWS UFFICIALEAggiornata 2 anni fa
The S3 file was uploaded successfully. It is a CSV file of file size 49.9MB. The Lambda file was not invoked and I did not see any entry in Cloud Watch logs. When the file size was reduced (<8MB) and uploaded, the next second, I got an entry in Cloud Watch logs and my Glue job got triggered immediately. "how the S3 trigger was created" -- Are you asking for any specific setting in it? Below are the settings we have provided (It is done by Admin and I do not have any control on these trigger setting): Bucket arn, Event Types (PUT, POST, COPY), Notification Name, Prefix (folder1/folder2/), Service Principal, Source Account, Statement ID (ddh-dms-glue-roles-DDHGlueLambdaPermission-asdeftgyhPUi)