Skip to content

Step Function X-Ray traceid not passed to ECS runTask

0

I want to user X-ray to trace my step function that has multiple steps using ECS runTask. But the trace id for the step is not passed on to ECS. The documentation for Step Functions suggests this should be possible with additional instrumentation and points to this, which in turn points to the Manual Instrumentation for Traces and Metric in Java page.

But the Otel integration can take the TraceId only from a HTTP request (which we don't have) or an env var (which the Step Function doesn't set, unlike for Lambdas where this works).

So does anybody know how to get the correct trace id for the current step from the Step Function so I can manually inject it into the task?

2 Answers
5
Accepted Answer

My understanding is also, that the Step Functions Context Object ($$) does not contain the X-Ray Trace ID, making it impossible to extract it natively within the ASL definition.

While Step Functions automatically passes the Trace ID to downstream Lambdas, it only passes it to the ECS Control Plane API (RunTask) during an ECS integration, failing to inject it into the actual container runtime/OS of your ECS/Fargate task.

To bridge this gap and maintain trace continuity, you can use one of these two proven production workarounds:

Option 1: Lambda Launcher Pattern

Instead of calling ECS directly from Step Functions, route the call through a lightweight Lambda function. Lambda automatically receives the _X_AMZN_TRACE_ID environment variable from Step Functions.

  1. Step Function invokes a Lambda.
  2. Lambda reads process.env._X_AMZN_TRACE_ID.
  3. Lambda triggers ecs.runTask via the AWS SDK, explicitly passing the trace ID into the container overrides:
// Inside your Lambda Launcher
const command = new RunTaskCommand({
    cluster: "my-cluster",
    taskDefinition: "my-task-def",
    launchType: "FARGATE",
    overrides: {
        containerOverrides: [
            {
                name: "my-app-container",
                environment: [
                    {
                        // The AWS X-Ray SDK inside ECS automatically picks this up
                        name: "_X_AMZN_TRACE_ID", 
                        value: process.env._X_AMZN_TRACE_ID
                    }
                ]
            }
        ]
    }
});

Option 2: Inject via State Machine Input ($)

If your Step Function is triggered by an upstream component that already knows the Trace ID (like an API Gateway Lambda), pass the Trace ID explicitly as part of the initial state machine execution Input ($).

Since it now resides in the normal payload JSON, you can natively map it into your ASL task definition overrides:

"ContainerOverrides": [
  {
    "Name": "my-app-container",
    "Environment": [
      {
        "Name": "_X_AMZN_TRACE_ID",
        "Value.$": "$.incomingTraceId" 
      }
    ]
  }
]

Note for your Container: Once the ID reaches ECS via environment variables, ensure your application's X-Ray or OpenTelemetry SDK is configured to pick up that variable (AWS X-Ray SDK looks for _X_AMZN_TRACE_ID by default) to initialize it as the parent context.

EXPERT

answered 3 months ago

-1

To pass the X-Ray trace ID from Step Functions to ECS tasks, you need to manually extract and inject the trace ID since Step Functions doesn't automatically set environment variables for ECS tasks like it does for Lambda functions.

The key is to use the traceHeader parameter when calling ECS RunTask from your Step Function. You'll need to construct the trace header in the format Root={TRACE_ID};Sampled={1 or 0} and pass it as part of your ECS task configuration.

In your Step Function state definition, you can access the trace ID from the execution context and pass it to your ECS task as an environment variable or container override. The trace header follows a specific format where you need to extract the trace ID from the current execution's X-Ray trace data.

For the ECS task itself, you'll need to manually instrument your application to read this trace ID (either from an environment variable you pass or from task metadata) and use it to initialize your X-Ray or OpenTelemetry tracer. This ensures continuity of the trace from Step Functions through to your ECS task execution.

The challenge you're facing is that unlike Lambda, where the _X_AMZN_TRACE_ID environment variable is automatically set, ECS tasks require you to explicitly pass this information through task parameters or environment variables when invoking RunTask from Step Functions.
Sources
Trace Step Functions request data in AWS X-Ray - AWS Step Functions

answered 3 months ago

  • In your Step Function state definition, you can access the trace ID from the execution context

    This is not correct. The Step Function execution context does not contain the trace ID.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.