Multiple lambda invocations for one event

The issue you're experiencing with multiple Lambda invocations for a single event is not uncommon and can occur due to various reasons. While your specific setup involves Google Cloud Storage and AWS Lambda, which is not a standard AWS integration, I can provide some general insights based on similar scenarios with AWS services.

1. Asynchronous invocations: When Lambda functions are invoked asynchronously (which is often the case with event-driven architectures), there's a possibility of multiple invocations for a single event. This is because Lambda may retry failed executions automatically.

2. Pub/Sub delivery guarantees: Google Cloud Pub/Sub, like many message queue systems, provides at-least-once delivery guarantees. This means that in some cases, the same message might be delivered more than once, leading to multiple Lambda invocations.

3. Network issues or timeouts: If there are network issues or if your function takes longer to process than expected, it might result in duplicate invocations.

4. Execution time spikes: Based on your description, you mentioned seeing duplicate data when there are spikes in execution time. This could indicate that your function is timing out or encountering issues during these spikes, leading to retries.

To address this issue:

1. Implement idempotency: Design your Lambda function to be idempotent, meaning it can handle multiple invocations for the same event without causing unintended side effects. This often involves checking if the event has already been processed before performing actions.

2. Use a tracking mechanism: Implement a system to track processed events, such as using a DynamoDB table to mark events as processed. Before processing an event, check if it's already been handled.

3. Optimize function performance: Ensure your function runs efficiently to avoid timeouts or execution time spikes that might lead to retries.

4. Adjust retry settings: If possible, adjust the retry settings in your Pub/Sub configuration to reduce unnecessary retries.

5. Implement error handling: Proper error handling in your Lambda function can help prevent unnecessary retries for certain types of failures.

6. Monitor and log: Implement detailed logging in your function to track each invocation and its outcome. This will help you identify patterns and root causes of multiple invocations.

By implementing these strategies, you should be able to reduce unnecessary Lambda invocations and ensure more predictable behavior in your system.
Sources
Multiple Lambda Execution for one EventsBridge Trigger | AWS re:Post
Invoke - AWS Lambda
Invoking Lambda with events from other AWS services - AWS Lambda