Issue

My AWS Lambda function is experiencing Timeout errors, and the function's response has been confirmed as received by the client. How do I troubleshoot whether the external extension is causing the Timeout?

Resolution

Option 1: Check the PostRuntimeExtensionsDuration metric in CloudWatch

If invocation rate is once per minute or less, this is the simplest and fastest method requiring no additional configuration. You can identify whether the extension is the cause by comparing the CloudWatch PostRuntimeExtensionsDuration and Duration metrics against the function's configured Timeout value.

1.1. Background

• The PostRuntimeExtensionsDuration metric represents the extension execution time after the runtime has completed processing the invoke event. Specifically, it measures the time (in ms) from when the runtime completes code execution and calls the /next API to signal Lambda, until the last extension calls the /next API [1].
• The Duration metric represents the total execution time of the function, including both the runtime execution time and the subsequent extension execution time. In other words, Duration is approximately Runtime execution time + PostRuntimeExtensionsDuration.

1.2. Steps

1. Open the CloudWatch console.
2. In the navigation pane, choose Classic metrics, then search for your Lambda function name in the Browse tab.
3. Choose Lambda > By Function Name.
4. Select the PostRuntimeExtensionsDuration metric along with other metrics for your function.
5. Set the Statistic to Maximum for both PostRuntimeExtensionsDuration and Duration metrics to compare against the function's configured Timeout value.

1.3. Troubleshooting example

• (Assuming the function includes an external extension) If the Duration metric has reached the Timeout and the Error metric is present, but the PostRuntimeExtensionsDuration metric is not recorded, this means the runtime's code execution did not complete. In this case, you need to troubleshoot which phase (Init Phase or Invoke Phase) consumed the time. Refer to the following re:Post Knowledge Center Article.
• If the Duration metric has reached the Timeout but the function's response is confirmed as returned (via execution logs or the client), this means the runtime's code execution completed successfully, but the external extension did not finish execution before the Timeout was reached. In this case, it helps to check the function's execution logs for any anomalies in the runtime execution time. If the PostRuntimeExtensionsDuration metric shows abnormally high values compared to normal, you can determine that the extension execution contributed to the Timeout error.

1.4. Limitations

• When multiple invocations occur within a 1-minute window, it can be difficult to identify the PostRuntimeExtensionsDuration value for a specific invocation.
• If the function's execution logs do not contain detailed logs for the runtime and extension code execution, CloudWatch metrics alone cannot determine the exact runtime execution time, making it difficult to conclusively attribute the delay solely to the extension.

Option 2: X-Ray Trace analysis

Enabling X-Ray Active Tracing on the Lambda function can overcome the limitations of Option 1. Through X-Ray Trace analysis, you can precisely identify the runtime execution time and PostRuntimeExtensionsDuration for individual invocations using the metrics within the AWS::Lambda::Function segment.

2.1. Prerequisites: Enable X-Ray Active Tracing

1. Open the Lambda console.
2. Choose your function.
3. Choose Configuration, then choose Monitoring and operations tools.
4. In the Additional monitoring tools section, choose Edit, then under the CloudWatch Application Signals and AWS X-Ray section, select Enable for Lambda service traces.
5. Choose Save.

2.2. Steps

1. Open the Lambda console.
2. Choose your function.
3. Choose Monitor, then select View X-Ray traces under the View CloudWatch logs dropdown.
4. Set the Start date/time and End date/time to the time range when the invocation occurred, enter the Trace ID in the search field, and click the View trace button.
5. Once the Trace details page loads, select the AWS::Lambda::Function segment.
6. In the Annotations tab, check the following metric values:

Metric	Description
aws:responseLatency	The amount of time it took for the function to run (ms)
aws:runtimeOverhead	The amount of additional time it took for the runtime to finish (ms)
aws:extensionOverhead	The amount of additional time it took for the extension to finish (ms)
aws:responseDuration	The amount of time it took to deliver the response to the customer (ms)

2.3. Troubleshooting example

• The image below shows the Trace result when a Timeout error occurred due to the extension waiting for an external API response after the runtime returned, in a Lambda function with a 20-second (20,000 ms) Timeout.
• Using the metrics in the AWS::Lambda::Function segment, you can calculate the time elapsed from when the runtime completed execution and responded to Lambda until it called the /next API:

aws:responseLatency   =   507.706 ms  (handler execution time)
aws:runtimeOverhead   =     0.459 ms  (runtime overhead)
aws:responseDuration  =     0.077 ms  (response delivery)
─────────────────────────────────────
Total                  =   508.242 ms 

• Additionally, the aws:extensionOverhead metric shows the time (in ms) from when the runtime called the /next API until the last extension called the /next API. The aws:extensionOverhead metric in the AWS::Lambda::Function segment corresponds to the PostRuntimeExtensionsDuration metric value, and Duration is approximately the sum of responseLatency + runtimeOverhead + responseDuration + extensionOverhead.

aws:responseLatency   =   507.706 ms  (handler execution time)
aws:runtimeOverhead   =     0.459 ms  (runtime overhead)
aws:responseDuration  =     0.077 ms  (response delivery)
aws:extensionOverhead = 19,491.991 ms (PostRuntimeExtensionsDuration)
─────────────────────────────────────
Total                  = 20,000.233 ms ≈ Duration (20,000 ms)

• In this example, the runtime completed in approximately 508ms and aws:extensionOverhead (19,492ms) consumed the majority of the configured Timeout value (20,000ms). This confirms that the execution delay in the extension, not the runtime, is the cause of the Timeout.

2.4. Limitations

• For functions using multiple external extensions, X-Ray Trace information alone cannot identify which specific extension consumed the most execution time.
• Additionally, there are limitations in analyzing the specific root cause within the extension's logic (e.g., external API response delay).
• X-Ray only samples a portion of invocations (1 request per second and 5 percent of additional requests). This means it may not always be possible to find a trace for the invocation in which the timeout occurred.
  • Important: The X-Ray sampling rate cannot be adjusted for Lambda functions [2].

Option 3: Extension debug logs and Lambda platform event log analysis

To overcome the limitations of Option 2, enable extension debug logging and configure the Lambda function's log format to JSON with the System log level set to DEBUG. This allows you to analyze which operations within the extension consumed time and examine the exact execution timing between the runtime and extension based on platform events.

3.1. Prerequisites

3.1.1. Enable extension debug logging

Some third-party extensions support a DEBUG log level. For example, for the Datadog Lambda Extension, setting the environment variable DD_LOG_LEVEL=debug on the function will record detailed internal operation logs to CloudWatch Logs. For more details, refer to the Datadog official documentation.

Important: After collecting sufficient data, disable debug logging to prevent unnecessary costs.

3.1.2. Configure system log level filtering

1. Open the Lambda console functions page.
2. Choose your function.
3. On the function Configuration page, choose Monitoring and operations tools.
4. In the Logging configuration pane, choose Edit.
5. Under Log content, confirm that Log format is set to JSON.
   • Important: System/application log level filtering can only be configured when the log format is set to JSON. It cannot be configured with the TEXT log format.
6. Use the radio button to set the System log level to DEBUG.
7. Choose Save.

After applying system log level filtering, platform events such as platform.start, platform.runtimeDone, and platform.report will be logged in JSON format in the function's CloudWatch execution logs.

3.2. Steps

1. Open the Lambda console functions page.
2. Choose your function.
3. Choose the Monitoring tab.
4. Choose View CloudWatch logs to open the CloudWatch console.
5. Scroll down and choose the log stream for the function invocation you want to examine [3].

3.3. Troubleshooting example

3.3.1. Extension debug log analysis

The logs below are an example from the Datadog Lambda Extension, showing the process of attempting to flush 1 trace buffered during the Invoke Phase to the Datadog endpoint.

DD_EXTENSION | DEBUG | Successfully buffered traces to be aggregated
DD_EXTENSION | DEBUG | Flushing 0 series and 1 distributions
DD_EXTENSION | DEBUG | TRACES | Flushing 1 traces
DD_EXTENSION | DEBUG | Sending with retry
DD_EXTENSION | DEBUG | Attempting request

The following describes the log patterns that differ depending on the success/failure/timeout of the flush attempt:

• A) If the flush attempt succeeds, you will see logs such as:

DD_EXTENSION | DEBUG | Successfully flushed ...

• B) If the flush attempt fails, you will see a response code such as 403 Forbidden from the Datadog endpoint, or DD_EXTENSION | ERROR extension error logs.
• C) If the function's configured Timeout is reached during the flush attempt, the pattern shows the platform.report platform event log appearing immediately without any additional DEBUG logs.

3.3.2. Lambda platform event log analysis

Lambda-generated platform event logs allow you to understand the internal behavior of the runtime and extension:

• platform.start event: Indicates that the function invocation has started, providing the approximate time of entry into the Invoke Phase. For warm starts, the execution log begins with this event.

{
    "time": "2026-07-24T09:25:47.590Z",
    "type": "platform.start",
    "record": {"requestId": "6f7f0961f83442118a7af6fe80b88d56"}
}

• platform.runtimeDone event: Indicates the point when the runtime called the /next API, meaning the runtime code execution has completed. Includes responseLatency, responseDuration, and runtimeOverhead metrics.

{
    "time": "2026-07-24T09:25:48.098Z",
    "type": "platform.runtimeDone",
    "record": {
        "requestId": "6f7f0961f83442118a7af6fe80b88d56",
        "status": "success"
        "spans": [
            {
                "name": "responseLatency",
                "start": "2026-07-24T09:25:47.590Z",
                "durationMs": 507.706
            },
            {
                "name": "responseDuration",
                "start": "2026-07-24T09:25:48.098Z",
                "durationMs": 0.077
            },
            {
                "name": "runtimeOverhead",
                "start": "2026-07-24T09:25:48.098Z",
                "durationMs": 0.459
            }
        ]
    }
}

• platform.report event: Indicates the invocation completion, providing the approximate time when the Invoke Phase ended. Includes precise millisecond values for Duration, Billed Duration, and the extensionOverhead metric.

{
    "time": "2026-07-24T09:26:09.598Z",
    "type": "platform.report",
    "record": {
        "requestId": "6f7f0961f83442118a7af6fe80b88d56",
        "metrics": {
            "durationMs": 20000,
            "billedDurationMs": 20000,
            "memorySizeMB": 256,
            "maxMemoryUsedMB": 83
        },
        "spans": [
            {
                "name": "extensionOverhead",
                "start": "2026-07-24T09:25:48.098Z",
                "durationMs": 19491.991
            }
        ],
        "status": "timeout"
    }
}

3.4. Considerations and limitations

• Configuring the log format to JSON may affect existing log parsing logic based on the Text format (CloudWatch Logs Insights queries, third-party log collectors, etc.), so apply carefully in production environments.
• Extension DEBUG logging can significantly increase log volume, so it must be disabled after collecting sufficient data.
• Important: The time field in platform event logs represents the time the event was recorded, which may not exactly correspond to the Timeout occurrence or shutdown completion time.

---

Related information

Lambda Telemetry API Event schema reference

Sending Lambda function logs to CloudWatch Logs

How do I troubleshoot Lambda function invocation timeout errors?

How do I use CloudWatch Logs to determine if my Lambda function timed out?