SQS FIFO queue, and lambda triggered

0

I have a FIFO SQS, and lambda triggered from that SQS, bulk size on lambda trigger is set to 1, lambda reserved concurrency is set to 1. and SQS visibility timeout is set to 2 minutes as the max amount of time for the lambda processed.

When 2 messages are sent to queue, 1 after the other, Lambda is triggered with 1 message, but the second message is delayed until the visibility timeout passed - 2 minuts. When new messages were sent to queue meanwhile, they were handled immediately.

The behavior I wish to achieve , is that when the lambda finished to handle the first message, a new lambda instance will handle the second message.

I tried changing the visibility timeout to 0, but as the lambda takes longer to execute, it handled the same message over and over.

已提問 3 個月前檢視次數 225 次
3 個答案
1
已接受的答案

The issue was that the messages had difference message group id. When the lambda finished handling the message with one group id, before the visibility timeout, and there was another message with another group id, this new message wasn't handled until the visibility timeout passed.

Different message group ids was not really needed, When I used constant message group id, all messages were handled 1 right after the other, even the visibility timeout didn't pass.

已回答 2 個月前
profile picture
專家
已審閱 2 個月前
0

When Lambda reserved concurrency is set to 1, then only one instance of the Lambda function will be invoked. As long as that Lambda function is functioning correctly, it will be the only Lambda function available to process messages. If that is the expected behavior, then only 1 message will be processed at a time from this queue.

When you send the 2 messages, do they have different message deduplication IDs?

AWS
已回答 3 個月前
  • They have different message body, and Content-based deduplication is Enabled. It's ok one message is processed at a time, but I expect that once message is done, the next one will start right away, while it start only once the visibility timeout passes. Can it be related to different message group ids for the messages?

  • Content-based deduplication is for an SNS FIFO topic subscribed by a SQS FIFO queue where SNS uses the message body to generate a unique hash value as the deduplication ID. If this isn't a SNS FIFO topic to SQS FIFO queue then you'll need to set the message deduplication ID yourself.

0

Does all the messages have the same Group ID? Lambda will never process messages from the same group ID at the same time.

Are you returning a valid response from the Lambda function? (verify in the logs that there are no errors). If you are not, the Lambda service will not delete the message from the queue, and when the visibility timeout expires, either Lambda will process the message again, or SQS will discard the message or move it to a DLQ, based on configuration (number of retries, DLQ). During the visibility timeout, Lambda will not handle messages with the same group ID.

profile pictureAWS
專家
Uri
已回答 3 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南