Use SSH keys properly on your VPS

Every D4 Networks VPS plan has an SSH Public Key field on the order form. Paste your public key there and the server is built with it already in place, so you never copy a key over a password login.

The root password still reaches you by email, and password logins stay enabled until you turn them off. So the server has two ways in, and the emailed one is the weaker of the two.

1. Make a key before you order

Run this on your own machine, not on a server. It works the same on macOS, Linux, and Windows, because Windows 10 and 11 include the OpenSSH client.

$ ssh-keygen -t ed25519 -a 100 -C "yourname@laptop"

Ed25519 is the type current OpenSSH picks by default, and it is the one to use. -a 100 controls how much work is needed to unlock the key with your passphrase; the default is 16, and raising it makes a stolen key file far slower to break into. -C adds a label so you can tell your keys apart later.

Press Enter to accept the default path, then set a passphrase when it asks. Without one, the key file works on its own, so anyone who copies it can log in as you.

Old RSA keys still work and our order form accepts them. Check the size with ssh-keygen -l -f ~/.ssh/id_rsa.pub and replace anything under 3072 bits. Physical security keys such as a YubiKey work too: ssh-keygen -t ed25519-sk makes a key that only works while the device is plugged in and you touch it.

2. Paste the public key on the order form

$ cat ~/.ssh/id_ed25519.pub

You get one line: the key type, a long scrambled section that is the key itself, then your label.

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJk2example0000example00 yourname@laptop

Paste all of it, on one line, with no line breaks. The form rejects anything that is not a real public key, which catches the two usual mistakes: pasting the private key, or pasting a fragment that a text editor wrapped across lines.

3. Log in and check the key was used

$ ssh [email protected]

If it asks for your key passphrase and not for a password, the key worked. Already ordered without pasting one? Add it now, using the emailed password once:

$ ssh-copy-id [email protected]

4. Turn off password logins

Until you do this, the emailed password is still a working way in for anyone who has it. SSH allows password logins unless you tell it not to.

See what your server is doing right now:

$ sudo sshd -T | grep -Ei 'passwordauthentication|permitrootlogin|kbdinteractive'

sshd -T reads every config file, works out the final answer, and prints that.

Here is where it usually goes wrong. The main config file is /etc/ssh/sshd_config, but there is also a folder beside it, /etc/ssh/sshd_config.d/, and every .conf file in there is pulled into the top of the main file. SSH then takes the first value it finds for a setting and ignores every later one, so a file in that folder wins against the main file. Editing the main file and seeing nothing change is the expected result.

$ ls /etc/ssh/sshd_config.d/

Write your own file with a name that sorts first, so it wins:

PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password

The second line matters as much as the first. Linux has a separate login system called PAM, which SSH can hand the login over to, and it asks for a password by its own route with its own switch. Turning off PasswordAuthentication alone can leave that prompt working. On AlmaLinux and Rocky the folder already holds 50-redhat.conf; Red Hat's documentation says your file must come first alphabetically, which 00- does.

$ sudo sshd -t
$ sudo systemctl restart ssh

$ sudo sshd -t
$ sudo systemctl restart sshd

The service is ssh on Debian and Ubuntu, sshd on RHEL-based systems. sshd -t checks the config and changes nothing, so run it before every restart.

5. Stop typing the passphrase

An agent is a small program that holds your unlocked key in memory, so you type the passphrase once per session:

$ ssh-add ~/.ssh/id_ed25519

macOS and most Linux desktops start an agent for you at login. Add AddKeysToAgent yes under Host * in the file below and the key loads on first use.

6. Give each server a short name

~/.ssh/config on your own machine turns a long command into a word:

Host nyc
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Host db
    HostName 10.20.0.5
    User deploy
    ProxyJump nyc
    IdentitiesOnly yes

ssh nyc now works on its own, and ssh db reaches a machine with no public address by connecting through the first server, which is what ProxyJump does. scp file.tar.gz nyc:/srv/ works the same way.

IdentitiesOnly yes matters once you have more than one key. Without it, ssh tries every key it can find, and the server allows six attempts before hanging up. With seven keys loaded you can be cut off before your correct key is tried.

7. When a good key gets rejected

Permission denied (publickey) with a key you know is correct is nearly always permissions. Before accepting a key, SSH checks who can write to your home directory and key files, and refuses without explaining if anyone but you can.

$ chmod 755 ~ && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys

If that is not it, ask the client what happened:

$ ssh -v [email protected]

Look for debug1: Offering public key. If your key is never offered, the problem is on your machine: the wrong IdentityFile, or too many keys. If it is offered and refused, the problem is on the server. One more quiet failure is a key pasted across two lines in authorized_keys, where neither half works.

Quick reference

Where to go next

With password logins off, no amount of guessing gets anyone in. Blocking repeat attempts is still worth doing, to quieten the logs and to cover services that cannot use keys, so pair this with fail2ban and a default deny firewall. On a brand new server, the first ten minutes on a new VPS runs through the whole baseline in one pass.