How to run a traceroute

A traceroute lists the routers between one machine and another and times each one. It is the first thing anyone asks for when a connection is slow or failing, and it is also the most misread output in hosting, so the reading section below matters more than the commands.

What a traceroute actually does

It sends a packet with a time-to-live of 1, then 2, then 3, and so on. Each router that receives a packet whose time-to-live has run out discards it and sends back an ICMP time-exceeded message, which is how it reveals itself. By default it does this three times per hop, which is why each line carries three timings.

Three consequences follow, and every misreading on this page comes from forgetting one of them:

• Every line is a different router answering a different packet. The rows are not repeated measurements of the same thing, and they do not add up.
• Replying at all is a courtesy. A router may rate-limit those replies, handle them at low priority, or refuse to send them.
• It maps the path out. The path back is invisible to it.

Windows

tracert is built in. Open Command Prompt or PowerShell.

tracert 203.0.113.10

Flags worth knowing:

To save the result as text you can paste into a ticket:

tracert -d 203.0.113.10 > traceroute.txt

Windows sends ICMP echo requests. There is no option to trace over TCP or UDP.

macOS

traceroute is preinstalled at /usr/sbin/traceroute.

traceroute 203.0.113.10

macOS sends UDP probes by default. To send ICMP instead, which is what Windows does and therefore what makes the two comparable:

sudo traceroute -I 203.0.113.10

Use traceroute6 for an IPv6 address.

Linux

traceroute is not on a minimal image. Install it first:

sudo apt install traceroute

sudo dnf install traceroute

The first is Debian and Ubuntu, the second Alma, Rocky, and RHEL. Most Debian and Ubuntu images already carry tracepath, which needs no install and no root and is a reasonable substitute:

tracepath 203.0.113.10

Linux traceroute also sends UDP by default. The two variants that answer different questions:

sudo traceroute -I 203.0.113.10

sudo traceroute -T -p 25565 203.0.113.10

The second sends TCP SYN packets to a port you name. Use it when the destination ignores ICMP, and use it when you want the trace to follow the same treatment your actual service gets, since some networks handle ICMP and game traffic very differently. Both need root.

Reading the output

This is where tickets are won and lost. A real-looking result:

  1     1 ms     1 ms     1 ms  192.168.1.1
  2     9 ms     8 ms     9 ms  192.0.2.1
  3    11 ms    10 ms    11 ms  192.0.2.34
  4     *        *        *     Request timed out.
  5   142 ms   139 ms   141 ms  198.51.100.7
  6    28 ms    27 ms    28 ms  198.51.100.62
  7    29 ms    28 ms    29 ms  203.0.113.10

Read from the bottom. The destination answered in 28 to 29 ms on all three probes. That is the connection. Hop 4 timing out and hop 5 reporting 142 ms are not faults, and the rest of this section explains why.

Only the last hop's latency describes your connection

The figure a player feels is the round trip to the server, which is the bottom line. Every line above it is a measurement of a router's willingness to talk about itself, taken on a different packet. A hop in the middle cannot make your connection slower than the number underneath it.

High latency at a middle hop is usually the router, not the path

When a router forwards your traffic it does so in hardware, at line rate. When it generates a time-exceeded reply it does so on its control processor, which is busy with routing protocols and treats this job as the lowest priority it has. A loaded core router can therefore answer late while forwarding perfectly.

The example proves it against itself. Hop 5 reported 142 ms, hop 6 reported 28 ms. A packet cannot reach hop 6 in 28 ms if it genuinely took 142 ms to get past hop 5. The 142 ms was never on the path.

The useful version of this rule: a middle hop's number only means something if the hops after it, and especially the last one, show it too. If the elevated latency carries all the way down to the destination, then the middle hop is telling you where the increase starts. If it does not carry down, ignore it.

Asterisks in the middle mean nothing on their own

A hop showing * * * while later hops answer is a hop that chose not to reply. Common reasons: the operator disables time-exceeded messages, the router rate-limits them, an MPLS segment carries your packet across several devices that never appear, or a firewall drops the probe but forwards real traffic. In every one of those cases your packets got through, which you can see from the fact that hop 5 onwards answered at all.

Asterisks at the last hop can be your own server's firewall

The destination not answering is worth checking, because it is the one line that matters. Before assuming a network fault:

• Windows blocks inbound ICMP echo by default. The server is up and will not say so. To allow it for diagnostics, in an elevated PowerShell on the server:

New-NetFirewallRule -DisplayName "Allow ICMPv4 echo request" -Protocol ICMPv4 -IcmpType 8 -Direction Inbound -Action Allow

• Linux may be configured to ignore echo requests. Check with sysctl net.ipv4.icmp_echo_ignore_all, where 1 means it is ignoring them.
• A UDP traceroute finishes when the destination answers a probe to a closed high port with a port-unreachable message. A firewall that drops that probe silently leaves the last line blank even though the host is fine. Trace to the port your service really listens on instead, with sudo traceroute -T -p 25565 203.0.113.10.
• Filtering in front of the server is a separate layer from anything you configure inside it. If you need to know whether a specific address was filtered, use Address lookup on the Attacks tab, described in When players cannot connect.

The path out is not the path back

Half of every round trip is the return journey, and a traceroute never shows it. Routing is not symmetric: the route from a player to your server and the route back can traverse completely different networks, and a fault on the return path shows up in your trace as nothing more than an inflated final number.

So a trace from the player to the server is half the evidence. Run the other half from the server back to the player's public IP address, at the same time, and keep both.

Hop names describe a naming convention, not geography

Reverse DNS strings on backbone routers are set by their operator, often contain airport or city codes, and are frequently stale or reused. Do not conclude your traffic detoured through another continent because a hop is named after one. A real detour shows up as a step increase in the final hop's latency, which is a number, not a string. Run with -d or -n and you will not be tempted.

Hops can change between probes

Networks spread traffic across parallel links, so consecutive probes to the same hop can be answered by different routers, and a hop can list two addresses. That is normal and is not a routing loop.

When a traceroute is the wrong tool

One traceroute is three packets per hop over a few seconds. A problem that affects a small percentage of packets, or appears for thirty seconds every few minutes, will very likely not be in that sample at all. For anything intermittent, run an MTR instead, which is the same idea sampled continuously.

What to send to support

• The traceroute from the affected machine to the server, as text.
• A traceroute from the server back to that machine's public IP address, taken in the same window.
• The destination address, and the port and protocol of the service.
• The time and time zone, and what the player actually experienced.

Paste the text rather than a screenshot: it can be searched, diffed against a later run, and forwarded. See Opening a ticket.

Related