How to wait to trigger Lambda function until specific files are uploaded

0

I'm new to AWS and working on Lambda function triggered by S3. I have two json files and need to use the both data to send queue to SQS. For example, two files, "file_a.json" and "file_b.json" are to be uploaded to S3 at the same time.

file_a.json
{"data": "123"}

file_b.json
{"data": "456"}

Then, Lambda will send queue message like ["123", "456"] which is created with the data from both files.

However, S3 invokes Lambda function every time the single file is uploaded and this makes it hard to send queue to SQS as I wrote above. I'm thinking if S3 can wait until the two files are uploaded to trigger Lambda, it can solve the issue. If anyone had faced the similar situation, I would like to know how to resolve this problem. Thanks!

2개 답변
3
수락된 답변

You will need to put this logic inside your Lambda function. Something like this:

If (my companion file exists in the bucket) then
    if (my file datetime is newer that my companion file) then
        parse files and add data to queue
    end
end
profile pictureAWS
전문가
kentrad
답변함 2년 전
profile picture
전문가
검토됨 일 년 전
profile pictureAWS
전문가
검토됨 2년 전
0

Hi, short Answer. That is not possible. The S3 Lambda Trigger sends an Event for every Object.

You can send the message to SQS and than trigger the Lambda function via the SQS Trigger. S3 -> SQS -> Lambda -> SQS The SQS Trigger offers a configurable Batch size and Batch window. That can work if you can guarantee that only these two files trigger the S3 Events in the defined Batch window.

AWS
Marco
답변함 2년 전
profile picture
전문가
검토됨 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠