Skip to content

AssumeRoleWithWebIdentity returns InvalidIdentityToken for a structurally valid Google Cloud service-account OIDC token

0

I'm federating a GCP service account into AWS via Workload Identity Federation (no static AWS access keys) using sts:AssumeRoleWithWebIdentity, and every attempt fails with:

  An error occurred (InvalidIdentityToken) when calling the
  AssumeRoleWithWebIdentity operation: The web identity token provided
  could not be validated.

Setup:

  • Custom OIDC identity provider in IAM for https://accounts.google.com (ClientIDList = ["sts.amazonaws.com"]).
  • IAM role trust policy: Federated = the provider ARN, condition accounts.google.com:aud == "sts.amazonaws.com" and accounts.google.com:sub == "<GCP service account uniqueId>".
  • Token minted via gcloud auth print-identity-token --impersonate-service-account=<sa> --audiences=sts.amazonaws.com (Google IAM Credentials API generateIdToken), and also reproduced from the token a Cloud Run service actually receives from its metadata server.

What I've already verified, so I can rule it out before anyone suggests it:

  • The token decodes to a structurally valid JWT: iss=https://accounts.google.com, aud=sts.amazonaws.com, sub/azp match the trust policy's condition exactly, exp/iat are always fresh (token minted immediately before each call). The signing key (kid) is confirmed present in Google's live JWKS (https://www.googleapis.com/oauth2/v3/certs) at the moment of the failed call.
  • Tested 3 structurally different trust-policy configurations against the same token: a custom OIDC provider, AWS's built-in Google federation principal (Federated: "accounts.google.com", no provider ARN) with its documented audazp / oaudaud condition-key mapping, and a plain azp condition on the custom provider. All three fail identically.
  • Tested the regional STS endpoint, the global endpoint, and explicit --endpoint-url for both — identical error every time.
  • Tested all three plausible thumbprint values from the live TLS chain of accounts.google.com:443 (leaf, intermediate, root CA) — no change.
  • Deleted and recreated the OIDC provider from scratch, and separately created a brand-new, never-before-used IAM role with an identical trust policy shape — both fail identically to the original, long-lived provider/role.
  • Repeated the entire test with a second, unrelated GCP service account (different sub, different token) trusted by its own fresh role — same error.
  • Ruled out networking/VPC/firewall on the GCP side: every failure is a complete, well-formed AWS STS error response with a RequestId, never a connection timeout or DNS failure, so the request is reaching AWS and being processed.
  • Confirmed no AWS Organizations SCP is involved (account isn't part of an Organization) and no duplicate/conflicting OIDC provider exists.

Given the token is independently verifiable as correctly signed, current, and correctly audienced/subjected, and the failure is consistent across every trust-policy/condition permutation I can construct (meaning it isn't a Condition-evaluation problem — InvalidIdentityToken occurs at token validation, a stage prior to Condition evaluation), what else can cause STS to reject a Google-issued web identity token at the validation stage? Is there a known additional requirement or limitation specific to Google Cloud service-account-issued ID tokens (as opposed to end-user Google Sign-In tokens) for AssumeRoleWithWebIdentity that isn't covered in the public documentation?

1 Answer
1

Hi Lucas,

Your write-up is unusually thorough, so I'll go straight to the part I think is actually happening, because I believe there is a confounding variable that invalidates one of your inferences. Then I'll cover what I was able to rule out by testing, and correct the other answer.

The other answer's mechanism is wrong, but it's pointing at the right area

The claim that STS requires the azp value to be registered in the OIDC provider's ClientIDList is not documented anywhere I can find, and it doesn't survive your own reasoning: the azp handling that AWS does document is purely a condition-key mapping, not a validation rule. From IAM and AWS STS condition context keys, for the default/Google mapping:

  • aud condition key ← the azp claim when azp is set, otherwise the aud claim
  • oaud condition key ← the aud claim

That mapping determines what accounts.google.com:aud reads, which is Condition evaluation. As you correctly noted, InvalidIdentityToken fires before that. So adding the SA unique ID as an audience for the reason given wouldn't be the fix.

But the general area, the provider's audience list, is where I'd look, for a different and much more specific reason.

The likely root cause: your custom OIDC provider changes the built-in Google path

On the AWS official knowledge center article Resolve the AWS STS error "InvalidIdentityToken", there is a community comment reporting a reproduction that matches your setup closely. Paraphrasing the reported steps:

  1. Create a role trusting Federated: "accounts.google.com" with only a accounts.google.com:sub condition. At this point AssumeRoleWithWebIdentity works.
  2. Then separately add an IAM identity provider: type OpenID Connect, Provider URL https://accounts.google.com, audience left empty.
  3. From that point on, calls fail with InvalidIdentityToken — with the trust policy completely unchanged.

The reporter's conclusion was that the fix is to add the audience you're actually using to the identity provider, and they noted it was surprising precisely because the trust policy hadn't changed.

If that behaviour still holds, it means an explicit OIDC provider registered for https://accounts.google.com becomes authoritative for that issuer account-wide, and the built-in Google path stops behaving independently.

This matters enormously for your test matrix. An IAM OIDC provider is an account-level resource, not a per-role one. So when you tested:

  • the custom OIDC provider,
  • the built-in principal Federated: "accounts.google.com" with the documented aud→azp / oaud→aud mapping,
  • a plain azp condition on the custom provider,

all three were evaluated while a custom OIDC provider for accounts.google.com existed in the account. Your provider deletions and recreations, and your fresh roles and second service account, all kept that one factor constant.

So your inference is half right. You're correct that this isn't a Condition-evaluation problem. But the conclusion that trust-policy shape is irrelevant doesn't follow from "all three permutations fail identically," because all three shared the same confounder. That's the one permutation your matrix doesn't contain.

The decisive experiment

Delete the OIDC provider for https://accounts.google.com entirely, confirm with aws iam list-open-id-connect-providers that no provider for that issuer remains in the account, then test only the built-in path with no provider ARN anywhere:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Federated": "accounts.google.com" },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "accounts.google.com:oaud": "sts.amazonaws.com",
        "accounts.google.com:sub": "<SA uniqueId>"
      }
    }
  }]
}

Note the deliberate use of oaud rather than aud here. Your token has aud=sts.amazonaws.com, and per the documented mapping the aud claim surfaces as oaud. Whether azp is present in your SA token determines what accounts.google.com:aud would contain, which is exactly why I'd avoid keying on aud for this first test.

If that succeeds, the provider was the problem and the answer to your question is "yes, there's an undocumented interaction, but it's about issuer registration rather than anything specific to service-account tokens."

The second thing to test: your audience value

To your direct question about whether there's a Google-specific requirement not in public docs, there is a strong hint in the docs themselves. The oaud key is described as: "It must be one of the OAuth 2.0 client IDs of your application." And aud is described as being set from the aud claim "for OAuth 2.0 Google client IDs of your application."

The entire built-in accounts.google.com model presupposes the audience is a Google OAuth 2.0 client ID, because it was designed for Google Sign-In, where your app has a registered client ID and users authenticate against it. A service-account ID token with aud=sts.amazonaws.com is an arbitrary string that Google will happily mint but that has no existence in Google's OAuth client registry.

Since generateIdToken lets you set any audience, test with an audience that is a real OAuth 2.0 client ID from your GCP project, and condition on that. If validation succeeds with a real client ID and fails with sts.amazonaws.com, you've found the constraint, and it's one that isn't stated plainly in the public documentation.

A documented cause of InvalidIdentityToken that is invisible in the trust policy

Worth a two-minute check because it would produce exactly your symptom. If the token contains an https://aws.amazon.com/roles claim and the requested role ARN does not match a value in it, STS returns InvalidIdentityToken regardless of the trust policy. This is documented under sts:RoleAuthorizedByIdp in the condition keys reference.

You almost certainly don't have that claim in a GCP SA token, but you listed the claims you verified rather than the full claim set, so dump every claim in the payload and confirm nothing unexpected is present.

What I ruled out for you by testing

The knowledge center article lists mechanical causes at the key-retrieval stage. I checked Google's live endpoints so you don't have to:

  • JWKS key count. STS supports a maximum of 100 keys in a JWKS. Google's https://www.googleapis.com/oauth2/v3/certs currently returns 5 keys, all RS256. Not the cause.
  • JWKS cacheability. If the JWKS returns Pragma: no-cache or Cache-Control: no-cache, STS won't cache it and can hit callback problems. Google returns cache-control: public, max-age=21277, must-revalidate, no-transform. Cacheable. Not the cause.
  • Discovery consistency. https://accounts.google.com/.well-known/openid-configuration reports issuer: https://accounts.google.com and jwks_uri: https://www.googleapis.com/oauth2/v3/certs, so the issuer matches your token's iss exactly and the key location resolves correctly.

One more thing that will save you effort: the same article notes that CloudTrail event history does not log this error, because it's treated as a client-side failure. So don't spend time looking for the failed calls there.

Escalation

You have RequestIds. Token validation happens inside STS and is not externally observable, so if the provider-deletion test above doesn't resolve it, this becomes a Support case with those RequestIds and a sample token. That's the only way to see which validation step rejected it. Do redact the token signature if you attach one.

If you need this to work reliably rather than just work

Stepping back from the immediate bug: accounts.google.com is a shared issuer. Every Google customer's tokens carry the same iss, which is why AWS special-cases it, why the aud/azp/oaud mapping is unusual, and why you've hit an undocumented edge. Building a production workload-federation path on it is fragile even once you get it working.

Two more durable options, depending on your constraints:

  • Amazon Cognito identity pools, which is the documented path for Google federation and handles the token exchange for you.
  • IAM Roles Anywhere, which uses X.509 certificates instead of OIDC. Heavier setup, but it's purpose-built for workloads outside AWS and has no shared-issuer ambiguity.

Either way, please post back what the no-provider test returns. If deleting the provider fixes it, that's a genuinely useful data point for the next person, because the current documentation does not say that registering an OIDC provider for accounts.google.com alters the built-in path.

AWS

answered a month ago

EXPERT

reviewed a month ago

  • was this resolved?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.