Bedrock AgentCore creation fails due to existing WorkloadIdentity

I use python boto3 to create a bedrock agentCore gateway:

client = boto3.client("bedrock-agentcore-control", region_name=region)
...
gw = client.create_gateway(
        name=props["GatewayName"],
        roleArn=props["RoleArn"],
        protocolType="MCP",
        protocolConfiguration={
            "mcp": {
                "searchType": "SEMANTIC",
            }
        },
        authorizerType="CUSTOM_JWT",
        authorizerConfiguration={
            "customJWTAuthorizer": {
                "discoveryUrl": props["DiscoveryUrl"],
                "allowedClients": [props["ClientId"]],
            }
        },
        kmsKeyArn=props["KmsKeyArn"],
        exceptionLevel="DEBUG",
    )

After around 8-10 minutes the status of the gateway is FAILED. CloudTrail shows the errors:

• 19:55:57 - CreateGateway succeeded
• 19:55:57 - CreateWorkloadIdentity succeeded
• 19:56:00 - CreateWorkloadIdentity FAILED - "already exists"
• 19:56:04 - CreateWorkloadIdentity FAILED - "already exists"
• 19:56:13 - CreateWorkloadIdentity FAILED - "already exists"
• 19:56:29 - CreateWorkloadIdentity FAILED - "already exists"
• 19:57:02 - CreateWorkloadIdentity FAILED - "already exists"
• 19:58:06 - CreateWorkloadIdentity FAILED - "already exists"

Also, I see some orphaned WorkloadIdentities, which I can't delete - "An error occurred (ValidationException) when calling the DeleteWorkloadIdentity operation: WorkloadIdentity is linked to a service and cannot be deleted by the caller. "

I've tried with the new gateway name. Still fails.