How to wait to trigger Lambda function until specific files are uploaded
I'm new to AWS and working on Lambda function triggered by S3. I have two json files and need to use the both data to send queue to SQS. For example, two files, "file_a.json" and "file_b.json" are to be uploaded to S3 at the same time.
file_a.json
{"data": "123"}
file_b.json
{"data": "456"}
Then, Lambda will send queue message like ["123", "456"] which is created with the data from both files.
However, S3 invokes Lambda function every time the single file is uploaded and this makes it hard to send queue to SQS as I wrote above. I'm thinking if S3 can wait until the two files are uploaded to trigger Lambda, it can solve the issue. If anyone had faced the similar situation, I would like to know how to resolve this problem. Thanks!
- 최신
- 최다 투표
- 가장 많은 댓글
You will need to put this logic inside your Lambda function. Something like this:
If (my companion file exists in the bucket) then
if (my file datetime is newer that my companion file) then
parse files and add data to queue
end
end
Hi, short Answer. That is not possible. The S3 Lambda Trigger sends an Event for every Object.
You can send the message to SQS and than trigger the Lambda function via the SQS Trigger. S3 -> SQS -> Lambda -> SQS The SQS Trigger offers a configurable Batch size and Batch window. That can work if you can guarantee that only these two files trigger the S3 Events in the defined Batch window.
