How do I troubleshoot issues when I connect to an API Gateway private API endpoint?

Short description

AWS resources in Amazon VPC might fail to connect to a private API endpoint for the following reasons:

• You didn't correctly configure the API Gateway resource policy for the private API endpoint.
• You didn't correctly configure the VPC endpoint policy for the interface VPC endpoint.
• You didn't correctly configure rules for an Amazon VPC security group.
• The private API's invoke URL doesn't include the correct DNS names.
• You didn't associate the private custom domain name with a VPC endpoint.
• You didn't correctly configure the resource policy for the private custom domain name.

To determine the cause, you can activate Amazon CloudWatch Logs for your API and review the logs.

If the API request doesn't produce logs after you activate CloudWatch Logs, then the request didn't reach the endpoint. If your API requests don't reach the endpoint, then check that you correctly configured the private API's invoke URL.
For more information, see How do I find API Gateway REST API errors in my CloudWatch logs?

Note: An incorrectly configured API Gateway resource policy or incorrect DNS name in the private API endpoint's invoke URL might cause connection issues.

Resolution

Confirm the cause of the issue

Complete the following steps:

1. Turn on CloudWatch Logs for your private REST API.
2. For Log level, choose INFO, and then choose Log full requests/responses data.
3. View your REST API's execution logs in CloudWatch to identify the cause of the issue.

If API requests reach the endpoint, then an error message that's similar to one of the following examples appears:

• "User: anonymous is not authorized to perform: execute-api:Invoke on resource:"
• "SSL: no alternative certificate subject name matches target host name custom-domain.com/resource"
• "Connection timed out"
• "Could not resolve host: https://example.com/resource"

"User: anonymous is not authorized to perform: execute-api:Invoke on resource:"

To resolve this issue, configure the following policies:

• The private API's API Gateway resource policy must allow traffic from the interface VPC endpoint or source VPC to the API endpoint.
• The VPC endpoint policy must allow client access to the private API endpoint. This is the resource policy for the execute-api service.
• The custom domain name's resource policy must allow access to your VPC endpoints to invoke your private custom domain name. By default, API Gateway provisions a custom domain name with an explicit deny in the custom domain name's resource policy. For more information, see Tutorial: Create and invoke a custom domain name for private APIs.
  Important: If you modify your API's resource policy, then deploy your API to the stage to save the changes.

"SSL: no alternative certificate subject name matches target host name custom-domain.com/resource"

To resolve this issue, check whether you created private custom domain name access associations for your private VPC endpoint for the execute-api service.

"Connection timed out"

To resolve connection timeout issues, complete the following steps:

1. Verify that an API Gateway execute-api VPC endpoint exists in the VPC where you host your client. Then, make sure that the endpoint is in the same AWS Region as the private API. If the endpoint doesn't exist, then create an interface VPC endpoint for API Gateway execute-api.

2. Use traceroute to verify that the client that invokes the private API exists in the same VPC or can access the VPC with the VPC endpoint.

   To install traceroute, run the following commands:

   Amazon Linux:

   sudo yum install traceroute

   Ubuntu:

   sudo apt-get install traceroute

   To use traceroute to test connectivity, run the following command:

   sudo traceroute -n -T -p 443 VPC-endpoint IP-address

   Note: The argument -T -p 443 -n performs a TCP-based trace on port 443. Replace VPC-endpoint IP-address with your VPC endpoint's IP address.

3. Verify that you correctly configured the rules for your Amazon VPC security groups.
   To test your Amazon VPC security groups, run the following command from the client that makes requests to the private API endpoint:

   telnet public-dns-hostname.execute-api.region.vpce.amazonaws.com 443

   Note: Replace public-dns-hostname with the public DNS hostnames that contain the VPC endpoint ID for your API. Also, replace region with the Region that your interface VPC endpoint is in.

   Or, run the following command from the client that makes requests to the private API custom domain name:

   telnet custom-domain-name 443

   Note: Replace custom-domain-name with your private API custom domain name.

If the connection times out, then you didn't correctly configure the rules for your Amazon VPC security groups.

The requesting resource must have a security group rule that allows TCP port 443 outbound traffic to the VPC endpoint's IP address range or security group. Also, the VPC endpoint must have a security group rule that allows TCP port 443 inbound traffic from the requesting resource's IP address range or security group.

For more information, see How do I restrict traffic to and from Amazon VPC resources?

"Could not resolve host:https://www.example.com/resource"

To resolve this issue, take the following actions:

• Verify that you associated the correct VPC to your Amazon Route 53 hosted zone.
• Verify that you created the record for your custom domain name in the Route 53 hosted zone with target as alias to your VPC endpoint DNS name.

Test that the private API endpoint's domain correctly resolves to the VPC endpoint's IP address

Note: Make sure that the client is within the VPC where the execute-api VPC endpoint exists.

Complete the following steps:

1. Run the following nslookup command from the client that makes requests to the private API endpoint:

   nslookup restapi-id.execute-api.region.amazonaws.com

   Note: Replace restapi-id with your private API's ID and region with the Region that your private API endpoint is in.
2. Run the following nslookup command from the client that makes requests to the private API custom domain name. :

   nslookup custom-domain-name

   Note: Replace custom-domain-name with your private API's custom domain name. A successful output shows the VPC endpoint's private IP addresses.
3. Run the following nslookup command:

   nslookup public-dns-hostname.execute-api.region.vpce.amazonaws.com

   Note: Replace public-dns-hostname with the public DNS hostnames that contain the VPC endpoint ID for your API. Replace region with the Region that your interface VPC endpoint is in. A successful output shows the VPC endpoint's private IP addresses.
4. Compare the IP addresses in the outputs of each command. If the IP addresses from each command output match, then the setup works as expected.

To activate private DNS for your VPC endpoint, complete the following steps.

1. Open the Amazon VPC console.
2. In the Endpoints pane, select your interface VPC endpoint.
3. Choose Actions.
4. Choose Modify Private DNS names.
5. Select Enable Private DNS Name, and then, choose Save Changes.

The API request doesn't produce any CloudWatch logs after you activate CloudWatch Logs

To resolve this issue, take the following actions:

• Correctly configure the private API endpoint's API Gateway resource policy.
• Correctly format your private API's invoke URL to access the private API endpoint.
  Note: If you activated a private DNS, then you must use endpoint-specific public DNS hostnames. If you didn't activate a private DNS, then use private DNS names.
• Map the private API custom domain name to the correct API stage.

Related information

How do I use an interface VPC endpoint to access an API Gateway private REST API in another account?

Why do I get an HTTP 403 Forbidden error when I connect to my API Gateway APIs from a VPC?

How do I monitor traffic in my VPC with flow logs?

How do I turn on CloudWatch Logs to troubleshoot my API Gateway REST API or WebSocket API?