How do I troubleshoot VPC-to-VPC connectivity through a transit gateway?

Short description

To troubleshoot connectivity between two Amazon VPC instances attached to the same transit gateway, check your network configurations. If you set up the configurations properly, then troubleshoot your connectivity issues.

Resolution

Check your network configurations

Check the configurations for the AWS Transit Gateway, the VPCs, and the Amazon Elastic Compute Cloud (Amazon EC2) instance.

Verify your security group configuration

Check if the security group and network access control lists (network ACLs) associated with the source elastic network interface or subnet allow necessary traffic.

Note: Security groups must allow traffic in only egress rules. However, network ACLs need to allow both outgoing and incoming traffic.

Confirm that the Amazon EC2 instance's security group and network ACL allows the traffic

Complete the following steps:

1. Open the Amazon EC2 console.
2. From the navigation pane, choose Instances.
3. Select the instance where you perform the connectivity test.
4. Choose the Security tab.
5. Check that the associated security group's Inbound rules and Outbound rules allowing the necessary traffic.
6. Open the Amazon VPC console.
7. From the navigation pane, choose Network ACLs.
8. Select the network ACL that's associated with the same subnet as the instance.
9. Select the Inbound rules and Outbound rules to verify that the rules allow the traffic.

Check if the source subnet VPC route table has a route that points towards destination CIDR block through the transit gateway transit gateway

Complete the following steps:

1. Open the Amazon VPC console.
2. From the navigation pane, choose Route Tables.
3. Select the route table used by the source instance.
4. Choose the Routes tab.
5. Verify that there's a route for the Remote VPC CIDR block under Destination. Then, verify that the Target is set to Transit Gateway ID.

Confirm that you attached the VPCs to the same transit gateway

If your connection uses one transit gateway, then complete the following steps:

1. Open the Amazon VPC console.
2. From the navigation pane, choose Transit Gateway Attachments.
3. Verify that the VPC attachments are associated with the same Transit Gateway ID.

Note: If you use multiple transit gateways or transit gateway peering, then verify that you attached the source and destination VPCs to their corresponding transit gateways.

Confirm that the Transit Gateway route table is associated with a VPC attachment

Complete the following steps:

1. Open the Amazon VPC console.
2. From the navigation pane, choose Transit gateway route tables.
3. Choose the route tables that are associated with the transit gateway VPC attachment of the source VPC.
4. Choose the Routes tab.
5. Verify that there is a route for Remote VPC IP range with Target as transit gateway VPC attachment that corresponds to the value for Remote VPC.
6. Choose the route tables that are associated with the transit gateway VPC attachment of the remote VPC.
7. Choose the Routes tab.
8. Verify that there is a route for Source VPC IP range with Target as VPC attachment. Verify that the route corresponds to the value for Source VPC.

Confirm that the network ACL associated with the transit gateway VPC subnets allows your traffic

Complete the following steps:

1. Open the AmazonEC2 console.
2. From the navigation pane, choose Network Interfaces.
3. In the search bar, enter Transit Gateway. All network interfaces of the transit gateway appear. Note the Subnet ID associated with the location where you created the transit gateway interfaces.
4. Open the Amazon VPC console.
5. From the navigation pane, choose Network ACLs.
6. In the search bar, enter the subnet ID that you noted in step 3 to find the network ACL associated with the subnet.
7. Confirm that the Inbound rules and Outbound rules of the network ACL allow the traffic. For details on configuring your network ACL rules, see Network ACLs for transit gateways in AWS Transit Gateway.

Check the Availability Zones for the transit gateway VPC attachment, the source and remote VPCs

When you attach a VPC to a transit gateway, you must specify one subnet from each Availability Zone. The subnets that you specify are the entry and exit points for your transit gateway traffic. For example, if your source is in us-east-1a, then you must configure the source and destination VPC attachments with at least one subnet in us-east-1a.

To check your Availability Zones, complete the following steps:

1. Open the Amazon VPC console.
2. From the navigation pane, choose Transit Gateway Attachments.
3. Choose the source VPC attachment.
4. Under Details, find the Subnet IDs. Verify that you selected a subnet from the source instance's Availability Zone.
5. Return to Transit Gateway Attachments. Then, choose the remote VPC attachment.
6. Under Details, find the Subnet IDs. Verify that you selected a subnet from the remote instance's Availability Zone.
7. To add an Availability Zone to a VPC attachment, choose Actions. Then, modify the Transit Gateway attachment and select any subnet from required Availability Zone.
   Note: Adding or modifying a VPC attachment subnet can impact data traffic when the attachment is in the Modifying state.

Troubleshoot connectivity issues

If the configuration is set up properly, then use Reachability Analyzer or Route Analyzer in AWS Network Manager to troubleshoot your connectivity issues. You can also monitor Amazon VPC Flow Logs and Amazon CloudWatch metrics to check for packet drops.

Troubleshoot connectivity issues with Amazon VPC Reachability Analyzer

Use Reachability Analyzer to analyze the path between your resource and the destination. For more information, see How Reachability Analyzer works.

Note: To analyze paths across multiple AWS accounts, activate trusted access for Reachability Analyzer with your organization from AWS Organizations. Reachability Analyzer supports cross-account analysis for only accounts inside your organization.

You can also use Amazon Q to use natural language queries to troubleshoot issues with Reachability Analyzer. For more information, see Introducing Amazon Q support for network troubleshooting (preview).

Troubleshoot connectivity issues with Route Analyzer

Prerequisite: Create a global network and register your transit gateway. For instructions, see Getting started with AWS Network Manager for Transit Gateways.

Complete the following steps:

1. Access the Amazon VPC console.
2. From the navigation pane, choose Network Manager.
3. Choose the global network where you registered your transit gateway.
4. From the navigation pane, choose Transit Gateway Network. Then, choose Route Analyzer.
5. Fill in the Source and Destination information as needed. Confirm that both Source and Destination have the same transit gateway.
6. Choose Run route analysis. Then, wait for Route Analyzer to complete the analysis.

Check the routing analysis. If your network's status is Not Connected, then Route Analyzer displays routing recommendations. Complete the recommendations and then rerun the test to confirm your network’s connectivity. For more information, see Diagnosing traffic disruption using AWS Transit Gateway Network Manager Route Analyzer.

Troubleshoot your issue with Amazon VPC Flow Logs and CloudWatch metrics

Prerequisite: Create AWS Transit Gateway Flow Logs for your transit gateway attachments or network interfaces. For network interfaces, for Log record format, choose Custom format, then include pkt-src-addr and pkt-dstaddr in the flow log record fields.

To use your transit gateway metrics to check if there are packet drops, complete the following steps:

1. Sign in to the AWS Management Console and open the Amazon CloudWatch console
2. Select All metrics.
3. Select VPC- Transit Gateway.
4. Select Per-attachment metric.

Then, review the Sum values for the following metrics for transit gateway attachments.

• The PacketDropCountBlackhole metric shows the number of bytes dropped because they matched a blackhole route on the transit gateway attachment.
• The PacketDropCountNoRoute metric shows the number of bytes dropped because they did not match a route on the transit gateway attachment.
• The PacketDropCountTTLExpired metric shows the number of packets dropped because the TTL expired.

Related information

Metrics and events in AWS Transit Gateway