Cannot Connect
SSH failures, for root and for named login users.
Start Here
ssh -v root@<cluster-ip>
The verbose output usually names the problem — and confirms which key was offered.
Check three things in the console before anything else:
| Check | Where |
|---|---|
| The cluster status is Running | Details tab |
| The IP you are using is the current one | Details → Connection Details, or Network & Security |
| A security group allows inbound TCP 22 | Network & Security → Security Groups |
Connection Refused or Timeout
| Cause | How to confirm | Fix |
|---|---|---|
| Port 22 is not open | Network & Security → Security Groups → expand the group → Inbound Rules | Attach a group with an SSH / TCP / 22 rule |
| The cluster is not Running | The status pill | Wait, or Reconfigure Cluster if it is stuck Creating |
| The IP changed | Compare with Connection Details | Use the current IP, then convert it to reserved |
| The login service is restarting | You just ran Restart Login Service or an image update | Wait a minute and retry |
| Your own network blocks outbound 22 | Try from a different network | Use your VPN, or ask your network team |
Provisioning succeeds and the cluster runs perfectly — you simply cannot log in. This is the single most common cause of "the cluster is broken". You do not need to recreate anything: attach a group that allows port 22 from Network & Security.
A Floating IP can change when the cluster's network resources are recreated, breaking every
~/.ssh/config entry and CI job that hard-codes it. One click on Convert to reserved removes
the whole problem.
Permission Denied (publickey)
The cluster is reachable; your key was not accepted.
1. Is the right key being offered?
ssh -v root@<cluster-ip> 2>&1 | grep -i 'offering\|Offering public key'
Be explicit:
ssh -i ~/.ssh/id_ed25519 root@<cluster-ip>
2. Is that key on the cluster?
Details → Connection Details → SSH Keys lists them by label. If the key you are using is not there, add it with Actions → Update SSH Keys.
Keys left out of the selection lose access. The dialog pre-selects the current keys — deselecting one removes it. Keep at least two keys on every cluster so one lost machine is not a lockout.
3. Local key permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
SSH refuses to use a private key that is group- or world-readable.
4. Are you the right user?
ssh root@<cluster-ip> # the cluster's own keys
ssh janedoe@<cluster-ip> # a Login User Management member
A member's key gets them in as themselves, not as root.
5. Too many keys offered
An agent with many keys can exhaust the server's attempt limit before reaching the right one:
ssh -o IdentitiesOnly=yes -i ~/.ssh/<the-right-key> root@<cluster-ip>
~/.ssh/config onceHost slurm-train
HostName <cluster-ip>
User root
IdentityFile ~/.ssh/id_ed25519
IdentitiesOnly yes
ServerAliveInterval 30
Then ssh slurm-train, with the right key, every time.
Host Key Verification Failed
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
Expected after the login node is recreated — a Restart Login Service, an image update, or a reconfigure. Remove the stale entry:
ssh-keygen -R <cluster-ip>
Then reconnect and accept the new key.
The message is also what a genuine interception looks like. If nothing on the cluster was restarted and the IP has not changed, do not clear the entry — investigate.
A Named User Cannot Log In
root works, a Login User Management member
does not.
| Cause | Check | Fix |
|---|---|---|
| No SSH key attached to the member | Login User Management → Members → the SSH Keys column | ⋮ → Update SSH Keys |
| Their group has no cluster access | Groups tab → Cluster Access column | ⋮ → Allow Access |
| They are in no group | Members filtered by group | ⋮ → Manage Members on a group with access |
| Login User Management is not enabled | The tab shows the Set up Login User Management empty state | Enable it |
| It was enabled moments ago | The panel shows an in-progress state | Wait for it to finish |
| The login name is wrong | The Login Name column | Use the exact name — lowercase, no typos |
ssh -v janedoe@<cluster-ip>
Permission denied (publickey) for a member usually means one of the first three rows above.
There is no password to issue or reset. If someone asks for one, they need their public key attached to their member record.
Adding a member does not grant them anything on its own. They need a group, and that group needs Allowed cluster access. This trips people up on the first user they add.
The Session Keeps Dropping
| Cause | Fix |
|---|---|
| Idle timeout on a network device | Add ServerAliveInterval 30 to ~/.ssh/config |
| The login service restarted | Expected — it recreates the pod and drops every session |
| An image update or reconfigure | Same |
| The login node ran out of resources | Someone is running heavy work on the login node instead of submitting it |
The login node is a submit host with no GPUs allocated to your work. Heavy processes there load it until everyone's sessions suffer, and a dropped connection or a login-service restart kills the process outright.
Use sbatch. A Slurm job survives disconnects, restarts and image updates; a shell does not.
tmux helps, but only within the pod's lifetmux or screen survives a dropped connection. It does not survive the pod being recreated by
a restart or an image update. For anything that must not be lost, it has to be a Slurm job.
Verify Once You Are In
sinfo # the cluster is scheduling
df -h | grep -E '/pfs|/shared' # your storage is mounted
echo test > /pfs/write-check && rm /pfs/write-check && echo writable
squeue # the queue is reachable
srun --help | grep -i container # container support is present