lambda function is executed twice without an additional trigger

Question

Hello: I have a lambda function triggered by a Detector Model (AWS IoT Events), which has been working fine for more than 6 months now.
I noticed now that in some cases, the lambda is being triggered twice (with a delay of ~70 seconds from the first execution) with an identical event passed from AWS IoT Events. Although I reviewed the logic and in fact added a variable that increments for each lambda call, looking at the Detector Model CloudWatch logs, I can definetely see that lambda is being involed only one time. So for some reason, lambda function is, in some cases, being (auto) triggered with same input data, around 70 seconds after the initial execution, duplicating the operation.
I really think this is an issue from the AWS infrastructure. I checked other similar posts but they might be related with a logic/implementation mistake. In this case, the logs confirm this is not the case.
I also wanted to open a Technical Case to report this, but it seems a paid support is required.
I would apreciate any help on this.

Didier_Durand · Accepted Answer

Hi,

This article give clear explanations and examples of why a Lambda may execute twice when only triggered once:

https://cloudonaut.io/your-lambda-function-might-execute-twice-deal-with-it/

So, the big recommendation for Lambda is to be idempotent: i.e. 2 executions must deliver same changes to permanent system state and other data as once.

See this post for idempotency: https://repost.aws/knowledge-center/lambda-function-idempotent

Best,

Didier

re:Post Agent · Answer

The behavior you're experiencing with your Lambda function being executed twice without an additional trigger is not typical, but it can occur due to certain aspects of how AWS Lambda operates.

First, it's important to understand that Lambda functions are designed to be idempotent, meaning they should be able to handle multiple executions with the same input without causing unintended side effects. This is because AWS cannot guarantee exactly-once execution of Lambda functions.

The scenario you're describing, where the function is executed again after about 70 seconds, could be related to Lambda's execution environment behavior. Lambda retains execution environments for a non-deterministic period after a function completes, allowing for potential reuse. However, there are operational factors in the Lambda service that can influence this retention time and potentially cause unexpected behavior.

It's possible that what you're seeing is related to Lambda's internal load balancing or scaling activities. Lambda is a highly available service that manages execution across multiple Availability Zones in an AWS Region. Depending on aggregate customer traffic, the service may load balance a function at any time. This can result in a function being invoked twice in a short period, with both executions potentially experiencing what appears to be a "cold start" due to this load rebalancing activity.

Additionally, Lambda's error handling and retry mechanisms could potentially cause a second execution if there was an internal error or timeout that wasn't visible to you.

While it's understandable to suspect an issue with AWS infrastructure, it's also worth considering that this behavior might be by design to ensure high availability and fault tolerance. However, if this is causing issues with your application logic, you may need to implement additional idempotency checks within your function to handle these potential duplicate executions.

If the issue persists and significantly impacts your application, you might consider implementing a deduplication mechanism using a unique identifier for each event, or exploring AWS Step Functions for more controlled execution flow.

Without access to AWS paid support, you could try posting your issue on AWS forums or the AWS subreddit for community assistance. You could also try adjusting your function's timeout settings or implementing more robust error handling to see if it affects the behavior you're observing.

**Sources**  
[Lambda execution environments - AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/execution-environments.html)  
[Scaling and concurrency in Lambda - AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/scaling-concurrency.html)

lambda function is executed twice without an additional trigger

Relevant content