Install SSH on Rocky Linux 8 – Quick guide to get the daemon running and connect from another machine
If you’ve just spun up a minimal Rocky Linux 8 install (or a container) and the ssh command screams “connection refused,” this walkthrough will show you how to put OpenSSH on the system, make sure it starts on boot, and verify that you can actually log in from elsewhere.
Update before you tinker
Running an upgrade first isn’t just good etiquette; it prevents nasty dependency surprises when the SSH packages pull in newer libraries.
sudo dnf upgrade --refresh -y
The --refresh flag forces DNF to re‑download the metadata, so you’re not stuck with a stale cache that could miss a critical security fix.
Do we already have the server?
Rocky Desktop ships OpenSSH by default, but a minimal install or a stripped‑down container usually does not. Check first – it saves you an unnecessary install and a reboot.
rpm -qa | grep openssh-server
If you see something like openssh-server-8.4p1-5.el8.x86_64 the daemon is already present; otherwise, move on to the next step.
Install the server (and client if you need it)
The server package provides sshd, the background service that listens for incoming connections. The client lives in a separate RPM; grab it only if this box will also be reaching out to other SSH hosts.
sudo dnf install -y openssh-server
# optional, handy for pulling files from elsewhere
sudo dnf install -y openssh-clients
I once tried to scp a file from a fresh Rocky VM and got “ssh: command not found” – installing the client saved me an hour of head‑scratching.
Enable and start the daemon
You want sshd alive now and after every reboot. The --now flag does both in one go, which is nicer than running two separate commands.
sudo systemctl enable --now sshd
Enabling adds a symlink to the appropriate run‑level directory; starting actually launches the process. You can confirm it’s listening with:
sudo systemctl status sshd # or ss -tlnp | grep :22
If you see LISTEN on port 22, the daemon is up.
Open the firewall (most people forget this)
Rocky’s default firewalld blocks incoming traffic on non‑essential ports. If you skip this step you’ll still get “Connection refused” even though sshd is running.
sudo firewall-cmd --permanent --add-service=ssh sudo firewall-cmd --reload
The first command creates a permanent rule; the second reloads the active configuration without rebooting. I’ve been bitten by this more than once after applying a security patch that reset firewalld to defaults.
Test locally before you go remote
Before you point another laptop at your server, make sure you can log in from the same machine:
ssh localhost
You should be prompted for your password (or key if you’ve set one up). If it works here but fails from another host, double‑check your network routing and firewall rules.
Connect from a client machine
Now fire up any terminal on the remote computer and run:
ssh user@your-server-ip
Replace user with an account that exists on the Rocky box and your-rocky-ip with its actual address (you can find it with ip addr show). If you get a “host key verification failed” warning, just type yes to accept the fingerprint – it’s normal the first time.
A quick note on security: If this server will be exposed to the internet, consider disabling password authentication and forcing key‑based logins. It’s not a huge hassle: edit /etc/ssh/sshd_config, set PasswordAuthentication no, then restart sshd. I’ve left passwords enabled on my home lab because I’m the only user, but for any production box that’s a recipe for brute‑force headaches.
That’s it – you now have a functional SSH service on Rocky Linux 8. Go ahead and start tinkering with remote commands, copy files over scp, or tunnel traffic as needed.
