How to Set Up SSH Key Authentication on Linux

SSH key authentication lets you log in to a Linux server by proving that you own a private key. The server stores only the matching public key. This is safer than depending on a reusable account password, and it also makes daily server access easier.

In this guide, I will show you how to set up SSH key authentication on Ubuntu, Debian, and other Linux systems. We will create an Ed25519 key on your computer, copy the public key to the server, test the login, fix common permission errors, and then safely disable SSH password authentication.

Important: Keep your current SSH session open until a second terminal can log in with the new key. If possible, keep your hosting provider’s web console ready as an emergency access method.

How SSH key authentication works

An SSH key pair contains two files:

• Private key: stays on your trusted computer and must never be shared.
• Public key: can be copied to the server and added to the user’s ~/.ssh/authorized_keys file.

During login, the server checks whether you control the private key that matches an authorized public key. The private key is not sent to the server.

What you need

• A Linux server running OpenSSH.
• A normal server user with permission to log in.
• Temporary password access or another administrator who can install your public key.
• OpenSSH client tools on your computer.
• Provider console access or another recovery path before changing SSH settings.

Run the key-generation commands on your own computer, not on the remote server.

Step 1: Check for existing SSH keys

On Linux, macOS, or Windows PowerShell with OpenSSH installed, list the SSH directory:

ls -la ~/.ssh

Common key names include id_ed25519, id_ed25519.pub, id_rsa, and id_rsa.pub. Files ending in .pub are public keys.

Do not overwrite a key unless you know where it is used. For a new server or project, a separate descriptive key name is easier to manage.

Step 2: Generate an Ed25519 SSH key

Ubuntu recommends Ed25519 because it provides strong security with a small key size and good performance. Create a new key on your computer:

ssh-keygen -t ed25519 -C "[email protected]"

The comment helps you identify the owner later. It does not need to be an email address.

To use a separate filename for this server:

ssh-keygen -t ed25519 -f ~/.ssh/linuxpanda_admin -C "linuxpanda-admin"

You will be asked for a passphrase. Use a strong passphrase for a human administrator’s key. It protects the private key if the file or computer is stolen. Automation keys need a separate risk review and should be limited to the smallest possible permissions.

If Ed25519 is unavailable because of an older compatibility requirement, create a 4096-bit RSA key:

ssh-keygen -t rsa -b 4096 -C "[email protected]"

Step 3: Confirm the key files

For the default Ed25519 filename, you should now have:

~/.ssh/id_ed25519
~/.ssh/id_ed25519.pub

id_ed25519 is private. Never paste it into a website, ticket, chat, or server terminal. id_ed25519.pub is the public key and can be installed on the server.

Display the public key when needed:

cat ~/.ssh/id_ed25519.pub

Step 4: Copy the public key with ssh-copy-id

The easiest method on Linux and macOS is ssh-copy-id:

ssh-copy-id username@server_ip

For a key with a custom filename, use -i:

ssh-copy-id -i ~/.ssh/linuxpanda_admin.pub username@server_ip

Enter the server user’s current password when prompted. The command adds the public key to the correct authorized_keys file.

If SSH uses a custom port:

ssh-copy-id -i ~/.ssh/linuxpanda_admin.pub -p 2222 username@server_ip

Copy an SSH public key manually

If ssh-copy-id is unavailable, log in to the server and create the required directory and file:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Copy the full, single-line public key from your computer and append it to ~/.ssh/authorized_keys on the server. Do not copy the private key.

Check ownership after installing it:

chown -R "$USER":"$USER" ~/.ssh
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Run these commands as the target user. If an administrator installs the key for another account, replace $USER with that account’s real name and path.

Step 5: Test SSH key login

Keep the original SSH window open. In a second terminal, test the login:

ssh username@server_ip

For a custom key:

ssh -i ~/.ssh/linuxpanda_admin username@server_ip

You may still be asked for the key’s passphrase. That is normal and is different from the server account password.

After login, confirm the account and its sudo access if administration requires it:

whoami
id
sudo -v

Use ssh-agent for a passphrase-protected key

ssh-agent can remember the unlocked key for your session, so you do not type the passphrase for every connection:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/linuxpanda_admin

Desktop key stores can also integrate with OpenSSH. Use the method supported by your operating system and protect the computer with disk encryption and a strong login.

Create an SSH config shortcut

If you connect often, add a host entry to ~/.ssh/config on your computer:

Host linuxpanda-server
    HostName 203.0.113.20
    User username
    IdentityFile ~/.ssh/linuxpanda_admin
    IdentitiesOnly yes

203.0.113.20 is a documentation-only example. Replace it with your server address. You can then connect with:

ssh linuxpanda-server

Step 6: Disable SSH password authentication safely

Do this only after every required administrator can log in with a tested key. Keep one working session open.

On Ubuntu, use a configuration snippet instead of mixing changes into the main file:

sudoedit /etc/ssh/sshd_config.d/00-key-auth.conf

Add these settings:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no

Do not set UsePAM no as a general hardening step. PAM handles more than simple password login on Ubuntu and disabling it can cause unexpected access and session problems.

Validate the file before reloading SSH:

sudo sshd -t
sudo sshd -T | grep -E 'pubkeyauthentication|passwordauthentication|kbdinteractiveauthentication|permitrootlogin'

If sshd -t prints an error, fix it before continuing. If the effective values are correct, reload the service:

sudo systemctl reload ssh

Test another new connection before closing any existing sessions. Remember that Ubuntu reads files in sshd_config.d in lexical order, and the first value found for most directives is used. The 00- prefix helps this snippet take precedence over later files, but you should always confirm the result with sshd -T.

Also allow the SSH port in your firewall before changing access. See our UFW firewall guide for Ubuntu.

Troubleshoot SSH public key authentication

Permission denied (publickey)

Use verbose client output:

ssh -vvv -i ~/.ssh/linuxpanda_admin username@server_ip

On the server, watch the SSH log from the working session:

sudo journalctl -fu ssh.service

Check the username, key path, public-key line, file ownership, directory permissions, and whether the server actually allows public-key authentication.

SSH offers the wrong key

Choose the key directly with -i and stop SSH from trying other agent keys:

ssh -o IdentitiesOnly=yes -i ~/.ssh/linuxpanda_admin username@server_ip

The key works, but SSH still asks for a password

It may be asking for the private-key passphrase rather than the server password. Read the prompt carefully. If the server falls back to password login, inspect ssh -vvv output and the server journal.

Root owns the user’s .ssh directory

Fix the ownership for the real target user:

sudo chown -R username:username /home/username/.ssh
sudo chmod 700 /home/username/.ssh
sudo chmod 600 /home/username/.ssh/authorized_keys

How to remove or rotate an SSH key

Each public key normally occupies one line in ~/.ssh/authorized_keys. To revoke access, remove the exact line for that key. Keep useful comments on keys so you can identify their owner and device.

For a safe rotation:

1. Create a new key on the trusted client.
2. Add its public key to the server.
3. Test the new key in a separate session.
4. Remove the old public key from authorized_keys.
5. Confirm the old key can no longer log in.

Disabling a Linux account password does not automatically remove SSH key access. Review the user’s authorized_keys file when offboarding an administrator.

Frequently asked questions

Is SSH key authentication passwordless?

It removes the need to send the server account password during login. A protected private key can still require its own local passphrase, which is recommended for human administrators.

Should I use Ed25519 or RSA?

Use Ed25519 for a normal modern OpenSSH setup. Use RSA 4096 only when compatibility with an older system requires it.

Can I use one SSH key for every server?

You can, but separate keys can make revocation and incident response easier. At minimum, avoid sharing one private key between different people.

Where is authorized_keys located?

For the default OpenSSH setup, it is ~/.ssh/authorized_keys inside the target user’s home directory.

Official references

• Ubuntu OpenSSH server documentation
• OpenSSH ssh-keygen manual
• OpenSSH sshd_config manual
• Ubuntu user-management security guidance

Conclusion

To set up SSH key authentication safely, create the private key on your trusted computer, copy only the public key to the server, test it in a second session, and validate the SSH configuration before disabling passwords.

SSH keys are one part of server security. Use a firewall, install security updates, limit administrator access, monitor login attempts, and keep tested backups. Continue with our Linux server hardening checklist.