Skip to content

403 to post_to_connection from lambda private-subnet

0

Hi A web socket API Gateway is configured. Connect, Authorize and SendMessage lambda functions are deployed in private subnet. Successfully connected and be able to send message to sendmessage lambda on public-api-gateway url.

When Sendmessage lambda boto3.apimanagemntapi.post_to_connect() is called it return 403. Both url types are tried VPCE: https://vpce-12345.execute-api.us-east-2.vpce.amazonaws.com/prod/ PublicURL: https://apiId.execute-api.us-east-2.amazonaws.com/prod/ Adding Host and X-Api-Gw-Api-Id headers also didn't make any difference.

Policy attached to role: { "Action": [ "execute-api:ManageConnections" ], "Condition": { "StringEquals": { "aws:sourceVpce": "vpce-12345" # With/WithOut has no difference } }, "Effect": "Allow", "Resource": "arn:aws:execute-api:us-east-2:<acct-id>:<apiId>/prod/POST/@connections/*" } In IAM Last Accessed Tab shows Amazon API Gateway agw-services-lambda-inline-policy Not accessed in the tracking period

SSL is not an issue I have noticed SSL successfully completes and when body is sent it is failing. Verified security groups and looked good. lambda-sg has no inbound rule but outbound has 443 0.0.0.0/0. vpce-sg has inbound 443 chained to lambda-sg, and outbound has no restriction.

Api-Gateway integration type is AWS_PROXY

Any pointers on how to fix this as part of private subnet execution. Thanks in advance

3 Answers
0

I've tried Private DNS option on/off in vpc-endpoint as well. With curl -vvv option I could see resolving to private address in both cases. So I am thinking the packet is reaching the endpoint. I ran it in EC2 instance to observe the curl logs.

In lambda handler to rule out ssl I have below and still see 403.

        apigwclient = boto3.client('apigatewaymanagementapi',
                                   config=my_config, # (region, v4)
                                   endpoint_url=enpurl,
                                   verify=False)

Having a hard time to find why wouldn't Api Gateway picks up the role?

Amazon API Gateway agw-services-lambda-inline-policy **Not accessed in the tracking period**

Trying with clean slate. VPC, Subnet, ApiGW, $connect and chatbotsender lambda functions assigned to private subnet

I see below error when called post_to_connection from Lambda function or curl. Please see the curl output below:

*   subjectAltName: "vpce-*.apigateway.us-east-2.vpce.amazonaws.com" matches cert's "*.apigateway.us-east-2.vpce.amazonaws.com"
* SSL certificate verified via OpenSSL.
* Established connection to vpce-*.apigateway.us-east-2.vpce.amazonaws.com (10.4.131.36 port 443) from 10.4.14.59 port 33142 
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://vpce-*.apigateway.us-east-2.vpce.amazonaws.com/production/@connections/eVoG1dfmCYcCJUQ%3D
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: *.execute-api.us-east-2.amazonaws.com]
* [HTTP/2] [1] [:path: /production/@connections/eVoG1dfmCYcCJUQ%3D]
* [HTTP/2] [1] [user-agent: curl/8.18.0]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [x-apigw-api-id: *]
* [HTTP/2] [1] [content-length: 32]
* [HTTP/2] [1] [content-type: application/x-www-form-urlencoded]
> POST /production/@connections/eVoG1dfmCYcCJUQ%3D HTTP/2
> Host: *.execute-api.us-east-2.amazonaws.com
> User-Agent: curl/8.18.0
> Accept: */*
> X-ApiGw-Api-Id: *
> Content-Length: 32
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 32 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 404 
< date: Tue, 02 Jun 2026 14:51:05 GMT
< content-length: 29
< x-amzn-requestid: 0e86be33-1e5e-4958-9d25-30358e236fab
< x-amz-apigw-id: eVoaDKq6iYcEbhQ=
< 
<UnknownOperationException/>

Reachability Test Results from Network Interfaces

# EC2 ENI to VPCE 
path2 nip-007a095f8ed6e0acf Reachable Succeeded eni-07e3cd31429e14244 <acctid> vpce-* 

# Lambda SG to VPC Endpoint Interface
path1 nip-0046d0dc1de867451 Reachable Succeeded eni-053a52353db141f2a <acctid>  vpce-*

Any pointers would be helpful. The endpoint policy in VPC interface is

	{
			"Sid": "VisualEditor1",
			"Effect": "Allow",
			"Principal": "*",
			"Action": [
				"lambda:InvokeFunction",
				"lambda:*"
			],
			"Resource": "*"
		},
		{
			"Effect": "Allow",
			"Principal": "*",
			"Action": [
				"execute-api:Invoke",
				"execute-api:ManageConnections"
			],
			"Resource": "arn:aws:execute-api:us-east-1:*:*/*/*/*"
		}

I know I switched to east-1 region now.

answered 8 days ago
  • Another connectivity test: Lambda connects to EC2 server on private address.

    def test_ec2_connect():
        ec2_private_ip = "private IP" 
        port = 8080 
    
        print(f"Attempting to connect to EC2 instance at {ec2_private_ip}:{port}...")
        try:
            with socket.create_connection((ec2_private_ip, port), timeout=5) as sok:
                print(f"Successfully connected to EC2 instance at {ec2_private_ip}:{port}")
        except Exception as e:
            print(f"Failed to connect to EC2 instance at {ec2_private_ip}:{port}. Error: {str(e)}")
    

    But when executing @connections/* it throws An error occurred (404) when calling the PostToConnection operation: <UnknownOperationException/>

0

Your Lambda is in a VPC with an API Gateway VPC endpoint + Private DNS. So even when you call the public url, DNS in the VPC silently that path, so post_to_connection returns 403.

answered 8 days ago
EXPERT
reviewed 8 days ago
0

fwiw: with three changes, the lambda function in private subnet is able to post messages to @connections/connectionId from private subnet

  1. Using NAT GW to route traffic out from the private lambda function.
  2. Since NAT is available, in VPC Interface endpoint's private DNS is switched off
  3. The role has exact resource for the ManageConnections except * at the end. arn:aws:execute-api:<region>:<acct-id>:<api-id>/dev/POST/@connections/*

I am still unsure vpc interface use here. It should work with Private DNS on, and route internally to see api-gateway endpoints with private DNS names.

answered 4 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.