Setting Up an Amazon CloudWatch Telemetry Pipeline to Ingest Okta SSO Logs

The Solution

Okta API Services App (public/private key pair)
    → JWT assertion (OAuth 2.0 client credentials)
        → AWS Secrets Manager  ({client_id, client_secret = RSA private key})
            → CloudWatch Telemetry Pipeline
                 source: okta_sso  →  processor: OCSF  →  sink: CloudWatch Logs
                    → Okta SSO events queryable in CloudWatch Logs

Prerequisites

• An Okta org with administrator access to create an API Services application
• An AWS account with permissions to create IAM roles, Secrets Manager secrets, and CloudWatch telemetry pipelines
• A target AWS Region (all resources — secret, role, pipeline — in the same Region)
• Familiarity with the AWS CLI, IAM policies, and CloudWatch Logs

Step 1: Create the Okta API Services application

In the Okta Admin Console:

1. Sign in to the Okta Admin Console.
2. In the left navigation, expand Applications and Resources → Applications.
3. Click Create App Integration.
4. In the Create app integration dialog, under Sign-in method, select API Services — this uses scoped OAuth 2.0 access tokens for machine-to-machine authentication.
5. Click Next.

Why API Services? The okta_sso telemetry pipeline source authenticates via the OAuth 2.0 Client Credentials (JWT assertion) flow, which requires an API Services application — not OIDC, SAML, or SWA. API Services apps sign in with a key pair rather than a user login.

Step 2: Name the app integration

On the next page, under App integration name, enter a unique, descriptive name that identifies this integration (for example, cloudwatch-okta-sso-ingestion). Click Save.

Step 3: Open the Public Keys editor

After saving, the application configuration page opens. This is where the app's client-authentication key pair lives — the private key signs the JWT assertion the pipeline sends to Okta.

1. On the General tab, scroll to the Public Keys section and click Edit.
2. Leave Configuration set to Save keys in Okta (the default). The key list is empty until you add one.
3. Click Add key to generate the key pair (next step).

Step 4: Generate the key pair and copy the private key

The Add a public key dialog opens.

1. Click Generate new key. Okta generates an RSA key pair and displays the public key (JWK) in the top panel.
2. In the Private key – Copy this! panel below, choose a format with the JSON / PEM toggle, then click Copy to clipboard.
3. Paste the private key somewhere safe immediately — it is shown only once. You will store it in AWS Secrets Manager as the client_secret value in a later step.
4. Click Done.
5. Back in the Public Keys section, click Save to attach the key to the app.

Handle the private key like a credential. It signs the JWT assertion the pipeline uses to authenticate to Okta. Anyone with it can impersonate this app. Never paste it into a ticket, chat, email, or commit — it goes only into Secrets Manager. If it leaks, delete the key in Okta and generate a new one.

Step 5: Set the client authentication method

Still on the app's General tab, in the Client Credentials section:

1. Click Edit.
2. For Client authentication, select Public key / Private key (not Client secret). This tells Okta to authenticate the app with the JWT assertion signed by the key pair from Step 4.
3. Click Save.
4. Copy the Client ID shown in this section — you will need it as the client_id value in Secrets Manager.

Step 6: Grant the okta.logs.read scope

The okta_sso source reads Okta system logs, which requires the okta.logs.read API scope.

1. Open the Okta API Scopes tab.
2. Locate okta.logs.read in the scope list (use the Not Granted filter to find it quickly).
3. In the Actions column, click Grant. Its Consent changes to Granted.

Step 7: Assign the Read-only Administrator role

The app needs an admin role to read log data at the org level.

1. Open the Admin roles tab and click Edit assignments.
2. Click Add assignment.
3. Under Role, select Read-only Administrator. (Okta notes this role cannot be further constrained and is active for the entire org.)
4. Click Save Changes.

The Okta side is now complete. You should have three things saved: the Client ID, the private key (from Step 4), and your Okta org URL (for example, https://<your-domain>.okta.com).

The AWS side (CLI)

Everything below uses the AWS CLI. Set these shell variables once and reuse them. All resources — secret, role, log group, and pipeline — must be in the same Region.

export REGION="us-east-1"
export ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text)"
export SECRET_NAME="okta-sso-credentials"
export ROLE_NAME="okta-sso-pipeline-role"
export LOG_GROUP="/aws/telemetry/okta-sso"
export PIPELINE_NAME="okta-sso-ingestion"
export OKTA_DOMAIN="<your-domain>.okta.com"     # your Okta org URL, without https://
export CLIENT_ID="<okta-app-client-id>"          # from Step 5

Step 8: Store the Okta credentials in AWS Secrets Manager

The pipeline reads two keys from a Secrets Manager secret: client_id and client_secret. Per the CloudWatch docs, client_secret is the RSA private key used to sign the JWT assertion (from Step 4) — not a password.

Build the secret with jq. If you copied the private key as JSON (JWK), save it as okta-private-key.json, then minify it to a single line before embedding it — this keeps the stored value compact and avoids formatting issues:

# 1. Minify the JWK private key to a single, compact line
jq -c . okta-private-key.json > okta-private-key.min.json

# 2. Assemble the secret payload (client_id + minified private key as client_secret).
#    --rawfile reads the minified key as a string; jq escapes it into the outer JSON
jq -n --arg cid "$CLIENT_ID" --rawfile pk okta-private-key.min.json \
   '{client_id:$cid, client_secret:$pk}' > secret.json

# 3. Create the secret
aws secretsmanager create-secret \
  --name "$SECRET_NAME" \
  --description "Okta API Services credentials for CloudWatch telemetry pipeline" \
  --secret-string file://secret.json \
  --region "$REGION"

# 4. Remove local files that contain the private key
rm -f okta-private-key.json okta-private-key.min.json secret.json

Private-key format: The docs specify client_secret as "the RSA private key used to sign the JWT assertion" but do not state an encoding. Okta lets you copy the key as JWK (JSON) or PEM. Validate which your pipeline accepts in a non-production run; if PEM is required, copy PEM in Step 4 and put the PEM string in client_secret instead of the JWK.

Source: CloudWatch pipelines configuration for Okta SSO

Step 9: Create the pipeline source (execution) role

The pipeline assumes an IAM role to read the secret. For an API-pull source like okta_sso, the role trusts the CloudWatch pipelines service principal telemetry-pipelines.observabilityadmin.amazonaws.com and needs secretsmanager:GetSecretValue on the secret.

# Trust policy — allows the CloudWatch pipelines service to assume the role
cat > trust-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

aws iam create-role \
  --role-name "$ROLE_NAME" \
  --assume-role-policy-document file://trust-policy.json

# Permission policy — read the secret (add kms:Decrypt only if the secret uses a customer-managed KMS key)
cat > role-permissions.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SecretsManagerAccess",
      "Effect": "Allow",
      "Action": ["secretsmanager:GetSecretValue"],
      "Resource": "arn:aws:secretsmanager:${REGION}:${ACCOUNT_ID}:secret:${SECRET_NAME}*"
    }
  ]
}
EOF

aws iam put-role-policy \
  --role-name "$ROLE_NAME" \
  --policy-name "$ROLE_NAME-policy" \
  --policy-document file://role-permissions.json

Caller (your) permissions. The identity that runs the next steps needs observabilityadmin:CreateTelemetryPipeline, iam:PassRole (to pass the role above), and logs:PutResourcePolicy (for Step 12).

Source: CloudWatch pipelines IAM policies and permissions

Step 10: Create the destination log group

CloudWatch pipelines does not create the log group — it must exist before you create the pipeline, in the same Region.

aws logs create-log-group --log-group-name "$LOG_GROUP" --region "$REGION"

Step 11: Create the telemetry pipeline

Define the pipeline configuration: okta_sso source (reading client_id/client_secret from the secret), the OCSF processor with the okta_sso schema, and the CloudWatch Logs sink.

# Write the pipeline config. The quoted heredoc ('EOF') is intentional: it stops the
# shell from expanding the ${{aws_secrets:...}} pipeline tokens. Edit the domain and
# secret name below by hand (they are literal here, not shell variables).
cat > pipeline.yaml <<'EOF'
source:
  okta_sso:
    domain: "<your-domain>.okta.com"
    range: "P7D"
    authentication:
      oauth2:
        client_id: "${{aws_secrets:okta-sso-credentials:client_id}}"
        client_secret: "${{aws_secrets:okta-sso-credentials:client_secret}}"
processor:
  - ocsf:
      version: "1.5"
      mapping_version: 1.5.0
      schema:
        okta_sso:
sink:
  cloudwatch_logs:
    log_group: "/aws/telemetry/okta-sso"
EOF

# Build the API input (Body is a string; jq --rawfile embeds the YAML safely)
jq -n --arg name "$PIPELINE_NAME" --rawfile body pipeline.yaml \
   '{Name:$name, Configuration:{Body:$body}}' > create-pipeline.json

# Create the pipeline — returns the pipeline ARN
aws observabilityadmin create-telemetry-pipeline \
  --cli-input-json file://create-pipeline.json \
  --region "$REGION"

Capture the Arn from the response — you need it for the resource policy in the next step:

export PIPELINE_ARN="arn:aws:observabilityadmin:${REGION}:${ACCOUNT_ID}:telemetry-pipeline/<pipeline-id>"

Notes grounded in the docs:

• range is an ISO 8601 duration (P7D = last 7 days). Optional; default is 0 hours, maximum 90 days.
• The OCSF processor for okta_sso must use version: "1.5" and mapping_version: 1.5.0 with the okta_sso: schema.
• The sink log_group must be an existing log group (a custom path — @original is only for cloudwatch_logs sources).

Sources: Okta SSO pipeline config · Parser processors (OCSF) · Sinks · create-telemetry-pipeline CLI

Step 12: Attach the CloudWatch Logs resource policy (within 5 minutes)

API-pull pipelines require a resource policy on the destination log group that lets the CloudWatch pipelines service principal write to it. You have less than 5 minutes after the pipeline is created — if the pipeline becomes active before the policy exists, data is dropped.

cat > resource-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com" },
      "Action": ["logs:CreateLogStream", "logs:PutLogEvents"],
      "Condition": {
        "StringEquals": { "aws:SourceArn": "${PIPELINE_ARN}" }
      }
    }
  ]
}
EOF

aws logs put-resource-policy \
  --region "$REGION" \
  --policy-name "resourceArn=arn:aws:logs:${REGION}:${ACCOUNT_ID}:log-group:${LOG_GROUP}:*" \
  --policy-document file://resource-policy.json

If a resource policy already exists on that log group, retrieve it with aws logs describe-resource-policies, add this statement to the existing Statement array, and re-run put-resource-policy with the merged document — do not overwrite it.

Source: Resource policies — CloudWatch pipelines IAM

Step 13: Verify ingestion

Confirm Okta events are arriving. The pipeline authenticates to Okta, reads the System Log, transforms events to OCSF, and writes them to your log group.

# Confirm a log stream was created
aws logs describe-log-streams \
  --log-group-name "$LOG_GROUP" \
  --order-by LastEventTime --descending --max-items 5 \
  --region "$REGION"

# Tail recent events (Ctrl-C to stop)
aws logs tail "$LOG_GROUP" --since 15m --follow --region "$REGION"

If nothing arrives, work the troubleshooting steps below. Note that authentication happens at pipeline activation and the source begins pulling within a few minutes of becoming active.

Troubleshooting

Symptom	Likely cause	Fix
Pipeline "Create failed — AWS Secrets Manager role has insufficient permissions" but GetSecretValue succeeds in CloudTrail	Secret is not a JSON object with client_id and client_secret	Rewrite the secret with both keys; client_secret is the RSA private key
No logs arrive, no errors	Resource policy missing or added after the 5-minute window	Recreate the resource policy (Step 12); if the window lapsed, recreate the pipeline then add the policy immediately
Genuine AccessDenied on GetSecretValue	Role permission or trust gap	Confirm secretsmanager:GetSecretValue on the secret ARN and trust for telemetry-pipelines.observabilityadmin.amazonaws.com
ValidationException on create	Malformed range or schema mismatch	Use an ISO 8601 duration (e.g. P7D); OCSF must be version 1.5 / mapping_version 1.5.0 with the okta_sso: schema
Auth fails at Okta	Missing scope or admin role	Grant okta.logs.read (Step 6) and assign Read-only Administrator (Step 7)

For a deeper walkthrough of the "insufficient permissions" case (where the IAM is fine and the secret format is the real cause), see the companion article: Troubleshooting the CloudWatch Telemetry Pipeline for Okta SSO: "Create failed — AWS Secrets Manager role has insufficient permissions."

References

• Source configuration for Okta SSO
• CloudWatch pipelines configuration for Okta SSO
• CloudWatch pipelines IAM policies and permissions
• Parser processors (OCSF)
• Sinks (CloudWatch Logs)
• create-telemetry-pipeline — AWS CLI reference
• Okta — Implement OAuth for Okta with a service app