SQS FIFO queue, and lambda triggered

0

I have a FIFO SQS, and lambda triggered from that SQS, bulk size on lambda trigger is set to 1, lambda reserved concurrency is set to 1. and SQS visibility timeout is set to 2 minutes as the max amount of time for the lambda processed.

When 2 messages are sent to queue, 1 after the other, Lambda is triggered with 1 message, but the second message is delayed until the visibility timeout passed - 2 minuts. When new messages were sent to queue meanwhile, they were handled immediately.

The behavior I wish to achieve , is that when the lambda finished to handle the first message, a new lambda instance will handle the second message.

I tried changing the visibility timeout to 0, but as the lambda takes longer to execute, it handled the same message over and over.

질문됨 3달 전225회 조회
3개 답변
1
수락된 답변

The issue was that the messages had difference message group id. When the lambda finished handling the message with one group id, before the visibility timeout, and there was another message with another group id, this new message wasn't handled until the visibility timeout passed.

Different message group ids was not really needed, When I used constant message group id, all messages were handled 1 right after the other, even the visibility timeout didn't pass.

답변함 2달 전
profile picture
전문가
검토됨 2달 전
0

When Lambda reserved concurrency is set to 1, then only one instance of the Lambda function will be invoked. As long as that Lambda function is functioning correctly, it will be the only Lambda function available to process messages. If that is the expected behavior, then only 1 message will be processed at a time from this queue.

When you send the 2 messages, do they have different message deduplication IDs?

AWS
답변함 3달 전
  • They have different message body, and Content-based deduplication is Enabled. It's ok one message is processed at a time, but I expect that once message is done, the next one will start right away, while it start only once the visibility timeout passes. Can it be related to different message group ids for the messages?

  • Content-based deduplication is for an SNS FIFO topic subscribed by a SQS FIFO queue where SNS uses the message body to generate a unique hash value as the deduplication ID. If this isn't a SNS FIFO topic to SQS FIFO queue then you'll need to set the message deduplication ID yourself.

0

Does all the messages have the same Group ID? Lambda will never process messages from the same group ID at the same time.

Are you returning a valid response from the Lambda function? (verify in the logs that there are no errors). If you are not, the Lambda service will not delete the message from the queue, and when the visibility timeout expires, either Lambda will process the message again, or SQS will discard the message or move it to a DLQ, based on configuration (number of retries, DLQ). During the visibility timeout, Lambda will not handle messages with the same group ID.

profile pictureAWS
전문가
Uri
답변함 3달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인