First steps on a new Linux server

A freshly built Linux server is a stock image with a root password, and it is reachable from the internet the moment it boots. This page is the short list of things worth doing before you install anything on it: update the packages, name it, set its clock, create an account that is not root, move to SSH keys, and look at what is already listening.

You need to be signed in as root over SSH before you start. Getting there is covered on Connecting to your server, and if SSH will not let you in at any point, Console access reaches the machine from the panel instead. Keep that page in mind: it is the escape hatch for the SSH changes near the end of this one.

Update the packages the image shipped with

An operating system image is a snapshot taken on the day it was built. Between that day and your server booting, some of the software on it has had security fixes published. Updating first means you are not installing your service on top of known-vulnerable libraries, and it gets the noisy part out of the way before you have anything running that an update could disturb.

On Debian and Ubuntu:

apt update
apt upgrade -y

On RHEL, AlmaLinux and Rocky Linux:

dnf upgrade -y

apt update refreshes the list of what is available and apt upgrade installs it. dnf refreshes its metadata on its own, so there is no separate step.

Some updates, a kernel in particular, only take effect after a reboot. Check whether one is pending rather than rebooting on reflex.

On Debian and Ubuntu, the file /var/run/reboot-required exists when a reboot is needed:

ls /var/run/reboot-required

On RHEL, AlmaLinux and Rocky Linux, install the utility that answers the same question and run it:

dnf install -y dnf-utils
needs-restarting -r

Reboot now if it says so, while the server is still empty. It is a much easier decision at this point than it will be once players are connected. Day-to-day updates, including doing them automatically, are on Installing and updating software.

Set the hostname

The hostname is the name the machine calls itself. It appears in your shell prompt, in every log line the system writes, in mail the server sends, and in the banner of anything that reports itself to a monitoring tool. The default is usually something generated, which is fine until you have three servers open in three terminal tabs and cannot tell which one you are about to reboot.

Both distribution families use the same command:

hostnamectl set-hostname game01.example.com

Add the same name to /etc/hosts so anything that resolves the local hostname gets an answer without a DNS lookup. Open the file and make sure a line maps a loopback address to the name you set, for example 127.0.1.1 followed by game01.example.com and game01. Software that fails to resolve its own hostname tends to fail slowly, with a delay on startup rather than an error, which is a miserable thing to debug later.

Log out and back in to see the new name in your prompt.

Set the timezone

Every log line on the server is stamped in the system timezone. When you are matching a crash in your game server's log against a graph in the panel or against a report from a player who says it broke at nine, those timestamps need to mean the same thing.

List the zones and set one:

timedatectl list-timezones
timedatectl set-timezone Etc/UTC

UTC is the pragmatic default when you run more than one server, because it never shifts for daylight saving and log timestamps from different machines line up without arithmetic. If you would rather read local time, set your own zone, for example Asia/Kolkata or Europe/Berlin. What matters is that you know which one is in use.

Then confirm the clock is being kept in sync rather than drifting:

timedatectl

Look for System clock synchronized: yes and an active NTP service. If it is off, turn it on with timedatectl set-ntp true. A drifting clock breaks TLS certificate checks and makes correlating logs across machines pointless.

Create a user that is not root

Root has no undo. A mistyped path in a rm command removes system files instead of a mistyped file, and any program you run inherits the ability to do the same. Working as an ordinary user with sudo in front of the commands that need it makes the destructive operations deliberate, and it gives you a second account to sign in with when something is wrong with the first.

On Debian and Ubuntu, adduser walks through creating the account, the home directory, and the password:

adduser deploy
usermod -aG sudo deploy

On RHEL, AlmaLinux and Rocky Linux, the equivalent is two commands, and the administrators' group is called wheel rather than sudo:

useradd -m -s /bin/bash deploy
passwd deploy
usermod -aG wheel deploy

Check the membership took effect before you rely on it:

id deploy

The group you added should be in the list. Full detail on groups, on granting sudo more narrowly than "everything", and on file ownership is at Users, groups and sudo.

Move to key-based SSH

A password can be guessed, and a server on a public address gets a continuous stream of automated attempts against root from the moment it boots. An SSH key cannot be guessed at any useful rate, is not typed anywhere it could be observed, and does not have to be remembered.

Creating a keypair on your own machine, copying it up with ssh-copy-id, and storing a public key in the panel for future builds are all covered on Connecting to your server and SSH keys. This section is the server-side half: how the key lands on the new account, and how to stop passwords being accepted without locking yourself out.

1. Put your public key on the new account

   From your own computer, the one-line version copies the key and sets the permissions for you:

   ShellCopy

   ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@203.0.113.10

   If you would rather do it on the server, the permissions are the part that matters, and so is the order. SSH ignores an authorized_keys file that other accounts could edit, and it does so silently, which looks exactly like a rejected key.

   Create the directory and the file first:

   ShellCopy

   mkdir -p /home/deploy/.ssh
   chmod 700 /home/deploy/.ssh
   touch /home/deploy/.ssh/authorized_keys

   Now paste the contents of your .pub file into /home/deploy/.ssh/authorized_keys, one key per line. Then set the permissions and hand the whole directory to the account, in that order, because a file you created as root stays owned by root until you do:

   ShellCopy

   chmod 600 /home/deploy/.ssh/authorized_keys
   chown -R deploy:deploy /home/deploy/.ssh

2. Prove it works before changing anything

   Leave your current session open. In a new terminal window, connect as the new user and run a command that needs sudo:

   ShellCopy

   ssh deploy@203.0.113.10
   sudo whoami

   You should get in without being asked for an SSH password, and sudo whoami should print root after you enter the account password. If either half fails, fix it now. Nothing below this step is safe until both work.

3. Turn off password authentication

   Once the key works, passwords are only a way in for someone else.

   One trap first, because it makes this look broken. On current Debian and Ubuntu images /etc/ssh/sshd_config starts with an Include /etc/ssh/sshd_config.d/*.conf line, and cloud images ship a drop-in in that directory setting PasswordAuthentication yes. SSH keeps the first value it reads, and the include is at the top, so a drop-in beats whatever you write in the main file.

   Because the first value wins, your file has to be read before the cloud image's. Name it so it sorts first, for example /etc/ssh/sshd_config.d/00-hardening.conf, containing:

   Copy

   PasswordAuthentication no
   PermitRootLogin prohibit-password

   Naming it 99- is the common mistake: it sorts after 50-cloud-init.conf, so the cloud image's PasswordAuthentication yes is read first and wins, and you are left believing passwords are off when they are still accepted.

   Never trust the file, check the value the server resolved:

   ShellCopy

   sudo sshd -T | grep -iE 'passwordauthentication|permitrootlogin'

   That prints the effective configuration after every include. If it does not say passwordauthentication no, your drop-in is losing to another one.

   If your /etc/ssh/sshd_config has no Include line for sshd_config.d, as on some older RHEL, AlmaLinux and Rocky images, set those two in /etc/ssh/sshd_config itself. PermitRootLogin prohibit-password keeps root usable with a key while refusing root password logins, which is the pairing automated attacks rely on.

   Then check what the server will actually use, drop-ins included, and restart. sshd -t prints nothing when the syntax is sound:

   ShellCopy

   sshd -t
   sshd -T | grep -iE "passwordauthentication|permitrootlogin"
   systemctl restart ssh

   The service is called ssh on Debian and Ubuntu, and sshd on RHEL, AlmaLinux and Rocky, so use systemctl restart sshd there. Restarting does not drop existing sessions. Open one more new session as deploy with your key, not as root with a password, which you have deliberately stopped accepting. Once that works, close the root session you have been holding.

Those two settings are the ones that matter on day one. Moving the port, restricting which accounts may sign in, and the rest of the sshd_config options are on Hardening SSH.

See what is already listening

Anything bound to a public address on your server can be reached by anyone. A stock image usually listens on more than you expect, because it ships with a mail transfer agent, a name resolution stub, or a database that a package pulled in as a dependency. You cannot decide what to shut down until you can see the list.

ss -lntup

The flags read as: -l listening sockets only, -n numeric ports rather than service names, -t TCP, -u UDP, -p the process that owns each socket. Run it as root, or with sudo, or the process column comes back empty for everything you do not own.

Read the Local Address:Port column first, because that is what decides exposure:

• 0.0.0.0:PORT or [::]:PORT means every interface, including the public one. Anyone who can route to your server can reach it.
• 127.0.0.1:PORT or [::1]:PORT means the loopback interface only. Nothing outside the machine can connect, which is where a database that only your own application uses belongs.

Go through the world-bound lines and decide, for each one, whether you meant it. For anything you did not, either stop and disable the service:

systemctl disable --now postfix

or reconfigure it to bind to 127.0.0.1 instead. Turning it off is covered in Managing services with systemd, which also explains the difference between stopping something now and stopping it coming back at the next boot.

Once you know what should be reachable, make sure the rules match on both sides: Using a firewall covers the one running on the server, and Port rules covers the network in front of it.

Before you install anything else

Two habits that are much cheaper to start now than to retrofit.

Know what your backups cover. Check what is included for your server on Backups, and keep your own copy of anything you would be unwilling to rebuild by hand, such as server configuration and player data.

Know how to start over. If a server gets into a state you cannot unpick, a reinstall is often faster than the investigation. See Reinstalling your server for what that destroys and what it keeps.

Related