Why do I get an "HTTP 504 Gateway Timeout" error from an outbound VPC connection to AWS services in Amazon Managed Grafana?

Short description

When Amazon Managed Grafana doesn't get a response from the upstream server in time to complete your request, you receive an HTTP 504 Gateway Timeout error. This commonly occurs when a workspace has an outbound VPC connection that uses public subnets. Because public subnets have a direct route to an internet gateway, this allows internet access to resources with a public IPv4 or IPv6 address. However, Amazon Managed Grafana gives private IPv4 addresses to its elastic network interfaces.

To resolve this error, use either of the following methods:

• Method 1: Configure an Amazon Managed Grafana outbound VPC connection to a private subnet.
• Method 2: Create a VPC endpoint for the corresponding service in the workspace's VPC.

Resolution

Method 1: Configure an Amazon Managed Grafana outbound VPC connection to a private subnet

1. Create subnets in at least two different Availability Zones in your AWS Region. The subnets must support IPv4.
2. Create a public NAT gateway. From the list of subnets, choose an existing public subnet.
3. Create a custom route table.
4. Associate the private subnets that you created in step 1 with the newly created route table.
5. Add a route to the route table with a destination of 0.0.0.0/0 and target of nat-gateway-id.
6. (Optional) Create a security group. Make sure that it allows connectivity to the data source through inbound and outbound rules. For example, if you want to connect to Amazon CloudWatch, then create an outbound HTTPS rule with a destination of 0.0.0.0/0.
7. Open the Amazon Managed Grafana console.
8. In the navigation pane, choose All workspaces.
9. Select the name of the workspace that you want to add a VPC outbound connection to.
10. In the Network access control tab, next to Outbound VPC connection, choose Edit to create your VPC connection.
11. Choose the VPC that you want to connect to.
12. Under Mappings, select the subnets that you created in step 1 in their relevant Availability Zones.
13. Under Security Groups, select at least one security group for this connection.
14. Choose Save changes.

Method 2: Create a VPC endpoint for the corresponding service in the workspace's VPC

1. Create an interface VPC endpoint that connects to the AWS service.
2. For VPC, select the VPC that's associated with the outbound connection for Amazon Managed Grafana.
3. For Subnets, select the subnets that are associated with the outbound connection for Amazon Managed Grafana.

To configure an outbound VPC connection for your workspace, complete steps 7-14 in Method 1.

Related information

How VPC connectivity works
Why do I get a 502 Bad Gateway Error when I am trying to connect to a data source after I configured the VPC in my Amazon Managed Grafana workspace?