- Newest
- Most votes
- Most comments
In the context of Amazon SQS and AWS Lambda, the "consumer" refers to the AWS Lambda function that is triggered by the SQS event and processes the messages from the queue.
When a message is received from the SQS queue, the Lambda function attempts to process it. If the processing is successful and the message is deleted from the queue, the process ends. However, if the Lambda function fails to process the message (due to an error in the function, for example), the message remains in the queue and the "ReceiveCount" for that message is incremented, until maxReceiveCount is reached, then going to DLQ.
Network errors that could cause a message to be moved from the source queue to the DLQ include issues with the network connectivity between the AWS services (SQS and Lambda), or within the Lambda function's execution environment. It's important to note that the SQS service itself is designed to be highly available and resilient to network errors, so network errors within SQS itself are rare.
If your lambda process multiple messages at a time, you should implement 'partial batch responses' so that not all messages are put back in the queue if one fails. Look for 'Implementing partial batch responses' in AWS Lambda SQS Doc, you will find some code samples.
Relevant content
- asked 2 years ago
- Accepted Answerasked 10 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated 5 months ago
Thank you for your reply. When you say "If the processing is successful and the message is deleted from the queue", is the message deleted manually by the lambda, or automatically by AWS when the lambda returns successfully (ie, does not throw an exception)?
When you implement partial batch responses, every message that are NOT in the "batchItemFailures" array are considered as proceed and removed from the queue. If your function throws an exception, the entire batch is considered a complete failure and messages are requeued after the visibility timeout.
One other question. Re "if the Lambda function fails to process the message (due to an error in the function, for example), the message remains in the queue and the "ReceiveCount" for that message is incremented, until maxReceiveCount is reached, then going to DLQ."
What causes the repeated incrementing of the ReceiveCount? Repeated attempts to get the lambda to process the message?