Diagnosing high latency

A method for finding where lag comes from, and for telling a slow route apart from a server too busy to answer on time.

Networking6 min read

On this page
  1. Get a number you can compare against
  2. Is it one player or everyone
  3. One player, or a handful on the same ISP
  4. Everyone in one country or on one network
  5. Everyone, everywhere, at once
  6. Test from a second network
  7. Check the server before blaming the network
  8. Routing latency against server-side lag
  9. When it looks like an attack
  10. What to record before asking for help
  11. Related

High latency has several causes that feel identical in game, and the fix for each one is useless against the others. This page is an order of operations for telling them apart, ending with the distinction that matters most: whether the delay is on the network or inside your server.

Get a number you can compare against

Without a baseline every measurement is an opinion. Before changing anything, collect two figures from the same machine.

The network round trip. From an affected player's machine:

ping -n 100 203.0.113.10
Shell
ping -c 100 203.0.113.10

Windows first, macOS and Linux second. Both print a summary with the minimum, average, and maximum at the end. Write down the average.

The in-game latency, from the game's own display, at the same moment.

Those two numbers measure different things, and the gap between them is the whole diagnosis. The ping is network time only. The in-game figure includes however long the server took to process the tick and reply.

Is it one player or everyone

This single question removes most of the possible causes, so ask it before you touch anything.

Is it one player, or everyone?One player, or a few onthe same ISPPoints toTheir connection or theirrouteThenWired, a quiet house, a phonehotspot, then an MTR each wayEveryone in one countryor on one networkPoints toRouting between that networkand the server's regionThenMTRs both ways from twoaffected players and one whois fineEveryone, everywhere, atoncePoints toThe server, or the link infront of itThenThe server checks first, thena test from a separatenetworkIs it one player, or everyone?One player, or a few on the same ISPPoints toTheir connection or their routeThenWired, a quiet house, a phone hotspot, thenan MTR each wayEveryone in one country or on onenetworkPoints toRouting between that network and theserver's regionThenMTRs both ways from two affected playersand one who is fineEveryone, everywhere, at oncePoints toThe server, or the link in front of itThenThe server checks first, then a test from aseparate network
One question splits the causes three ways, and each answer points somewhere different and calls for different evidence. The three subsections below take them in turn. Illustration.

One player, or a handful on the same ISP

Their connection or their route. Have them, in this order:

  1. Test wired instead of Wi-Fi. Wi-Fi adds latency and loss that no speed test shows.
  2. Ping the server while nothing else in the house is downloading, then again during a large download. A large gap is their own line saturating.
  3. Retest on a phone hotspot, which is a completely different ISP and a completely different path.
  4. Run an MTR to the server, and have you run one back to their IP address.

If the hotspot is fine and the home connection is not, it is their connection, and nothing you change on the server will help.

Everyone in one country or on one network

A routing problem between that network and the region the server is in. Collect MTRs both ways from at least two affected players, plus one from a player who is fine, so there is something to compare against. That set of reports is what a ticket needs.

Everyone, everywhere, at once

The server, or the link in front of it. Skip to the server checks below. If the server also looks clean, test from a machine on a completely separate network, such as another server you own, to rule out your own vantage point.

Test from a second network

One machine cannot tell you the difference between "the path is bad" and "my connection is bad". A second, unrelated vantage point can. Use a phone hotspot, another server, or someone in a different country.

Run the same ping and the same MTR from both. If the second network is clean, the fault is on the first one's path, and you now know which half of the route to investigate.

Check the server before blaming the network

This is the step people skip, and it is the one that most often holds the answer. A server that cannot keep up produces lag with a perfectly healthy network path underneath it.

Start with the live gauges on Server overview, then get more detail on the machine itself.

On Linux:

Shell
uptime

Three load averages over 1, 5, and 15 minutes. Compare them against the core count from nproc. A load average sitting well above the core count means processes are queueing for CPU, and everything they do arrives late.

Shell
top -b -n 1 | head -20

On the %Cpu(s) line, four fields decide the story:

FieldMeans
usTime in your own processes. An ordinary busy server.
syKernel time. Very high with moderate us often means packet processing.
waWaiting on disk. The server is stalling on storage, not on CPU.
stSteal time. The CPU was taken by the hypervisor. Nothing inside the server can fix that one.
Shell
free -h

If available is near zero and swap is in use, the server stalls in bursts that feel exactly like network spikes.

Shell
vmstat 1 5

Five one-second samples, which catches short spikes that a single top run misses.

On Windows, Task Manager's Performance tab covers the same ground, or from PowerShell:

PowerShell
Get-Counter '\Processor(_Total)\% Processor Time' -SampleInterval 1 -MaxSamples 10
30% in total, 100% where it mattersThe other three cores share the remaining 20 points between them.Total CPU, the figure nobody suspects30%Core 1: the game's world simulation100%Core 2Core 3Core 4025507510030% in total, 100% where it mattersThe other three cores share the remaining 20 pointsbetween them.Total CPU, the figure nobody suspects30%Core 1: the game's world simulation100%Core 2Core 3Core 40255075100
The note's example: four cores reading 30% in total while the one core running the world simulation is at 100%. The total averages the saturated core away. Derived from the example in the note above; the other three cores are drawn evenly.

Routing latency against server-side lag

With a ping figure and a server-load figure in hand, this table names the suspect.

What you observeWhere it points
Ping is high and in-game latency is high by about the same amountThe network path
Ping is normal and steady, in-game latency is high or spikyThe server or the game process
Ping is fine from you, high from one region onlyRouting for that region
Both rise at the same time every dayCongestion on the path, or your own peak hours
Both rise as the player count risesThe server, not the network
Ping is fine, but players time out on connect rather than lagNot latency. See the protection checklist below

The logic behind row two is worth stating plainly. If ping says 30 ms and players report 200 ms, no router added the other 170 ms. The packets are arriving on time and the server is answering late, which means the tick is running long.

Ping 30 ms, in game 200 msThe gap is the server, not the route.Ping: network time only30 msIn-game latency: the network, plus the server's reply30170, the server answering late200 ms050100150200Network timeThe server answering latePing 30 ms, in game 200 msThe gap is the server, not the route.Ping: network time only30 msIn-game latency: the network, plus the server's reply30170, the server answering late200 ms050100150200Network timeThe server answering late
The same example to scale. The ping is network time only, so the 170 ms between it and the in-game figure was added after the packets arrived, by the server answering late. Derived from the example above.

Most game servers expose their own timing under some name: a tick rate, a server thread time, a frame time, or a hitch warning in the console. Find the one your game uses and watch it while the ping runs in another window. A server whose tick is late lags exactly like a bad route, and no amount of MTR will ever show it, because the network is not the thing that is slow.

Common causes on that side, in rough order of frequency: too many plugins, scripts, or mods doing work every tick; a plugin doing disk or database work inside the main loop; too many entities or too much world activity; not enough CPU for the player count; and storage that has become slow, which shows as wa in top.

When it looks like an attack

Latency that jumps for everybody at once, with no change on your side and no matching rise in server load, is worth checking against the protection layer before you spend an evening on plugins.

  • The Attacks tab shows what was filtered and when. What is exact there and what is a sampled estimate is set out in Reading the Attacks tab.
  • Protection tuned too tightly looks like an outage rather than lag: players time out or cannot join at all, usually at your busiest hour. When players cannot connect works through those causes in order and is the right page if connections are failing rather than dragging.
  • Platform status reports whether the panel's own upstream systems are answering. It does not report on your server, so a green status does not clear anything here.

What to record before asking for help

  • Ping summaries, as text, from at least two networks, with the totals line.
  • An MTR each way, at least 300 cycles, taken while the problem was happening. See How to run an MTR.
  • The in-game latency figure alongside the ping figure from the same moment, since the gap between them is the evidence.
  • Your baseline from when it was fine.
  • The server's own load figures from the same window, and the game's own tick or thread timing.
  • Times, with a time zone, and whether it correlates with player count.

Send text rather than screenshots. See Opening a ticket.

On clouduxe.com. The pages on the main site that go with this guide.

Need a server, or a hand with one?

Pick a plan to deploy a server, or open a ticket in the panel. Support runs 24/7/365.