How do I read and troubleshoot my traceroute to resolve AWS Direct Connect issues?

Resolution

Run a traceroute

Run the traceroute command for your operating system. Replace example-destination-router with your destination router:

Windows:

tracert <example-destination-router>

MacOS:

traceroute <example-destination-router>

Linux:

traceroute <example-destination-router>

Read your traceroute output

When you run traceroute commands, a list of hops (network devices) that the packets are passing through is displayed. Each hop is listed with its IP address, hostname (if available), and the amount of time that the packet took to reach the hop in milliseconds. The output consists of a series of rows, with each row representing a hop along the network path.

The traceroute row consists of the following key components:

• Hop number: Indicates the sequence of the hop starting from one and incrementing with each subsequent hop. It also identifies the order that routers are encountered along the path.
• IP address or hostname: Identifies the IP address or hostname of the router at the specific hop that's displayed. IP addresses are shown in dotted decimal notation, and hostnames show descriptive information about the router.
• Round-trip time (RTT): Measures the time that it takes for a packet to travel from the source device to the router at the specific hop and then back. 

Example traceroute output:

traceroute to 203.0.113.5 (203.0.113.5), 30 hops max, 60 byte packets 
1  192.168.0.1 (192.168.0.1)  1.112 ms  0.948 ms  0.847 ms 
2  10.0.0.1 (10.0.0.1)  5.463 ms  *  5.385 ms 
3  203.0.113.5 (203.0.113.5)  20.846 ms  20.912 ms  21.156 ms 
4  *  *  *

The preceding example traceroute output contains the following information about the hops:

• Hop 1: The IP address is 192.168.0.1 and the RTT values (ms) are 1.112, 0.948, and 0.847. This hop displays low RTT values and indicates a fast and responsive connection to the associated router.
• Hop 2: The IP address is 10.0.0.1, the RTT values (ms) are 5.463, 5.385, and an asterisk is shown. This hop has slightly higher RTT values than hop 1, but are still low and indicate a fast and responsive connection. The asterisk indicates that a timeout occurred and the router didn't respond within an expected timeframe.
• Hop 3: The IP address is 203.0.113.5 and the RTT values (ms) are 20.846, 20.912, and 21.156. This hop has higher RTT values and indicates possible network congestion or longer processing times at the destination router.
• Hop 4: This hop shows three consecutive asterisks that indicate no responses were received from the associated router. The absence of responses at subsequent hops indicates a loss of connectivity or filtering of traceroute packets.

Note:

• Hops with low RTT values and no asterisks indicate a fast and responsive connection to the router.
• Hops with high RTT values indicate a slow connection and potential network congestion or longer processing times at the destination router.
• Hops that show an asterisk indicate that a timeout occurred and the router didn't respond within an expected timeframe. This can also indicate that the router at that hop is configured to not respond to traceroute packets. This is because of the firewall configuration, ICMP rate limiting, or router configuration.
• Hops that show three consecutive asterisks indicate that no responses were received from the associated router.

The presence of asterisks in the middle of hops doesn't always mean that there's a problem or network issue. Traceroutes continue to probe subsequent hops to gather information about the network path. If the output shows consistent asterisks throughout the traceroute, then this can indicate a network issue or a misconfiguration.

Troubleshoot your traceroute

High latency occurs when packets take too long to reach their destination. Hops with high RTTs cause high latency. If RTTs are consistently high across multiple traceroutes, then this indicates issues with your router or network.

To troubleshoot this issue, complete the following steps:

1.    Open a command prompt or terminal window on your local device.

2.    Type the command traceroute <IP address of the destination host>, and then press Enter.
Note: The output of the preceding command displays a list of hops that the packets are passing through on their way to the destination host. The RTT is also displayed.

3.    Open the AWS Management Console, and then identify the hop with the IP address of the Direct Connect endpoint.

4.    Review the traceroute output, and identify any hops that have a higher RTT than expected. These hops might be contributing to the latency issue.

5.    Check whether the hops with the latency issues are the Direct Connect endpoint.

6.    After identifying the hops that are causing latency issues, check for causes of congestion in the network configuration of each device in the patch.

7.    If the hop that's causing latency issues is the Direct Connect endpoint, then contact AWS Support.