Connect to a Linux GPU node

Connecting to a Linux GPU node is the same SSH flow as any Linux node. The only addition is confirming the card once you are in; for the full connect reference, finding the IP, password versus key, host fingerprints, see Connect to a Linux node.

Connect over SSH

ssh <username>@<gpu-node-ip>

Or with an SSH key:

chmod 600 ~/.ssh/<private-key-file>
ssh -i ~/.ssh/<private-key-file> <username>@<gpu-node-ip>

The default user, password retrieval, and security group rules are identical to a regular Linux node.

Check the GPU with nvidia-smi

Run nvidia-smi as your first action after login:

nvidia-smi

A healthy node returns the card name, driver version, CUDA version, memory, and utilization. On a single-card L4 node, the top of the output looks like this; your card, driver, and numbers will differ:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.82.07              Driver Version: 580.82.07      CUDA Version: 13.0     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA L4                      Off |   00000000:00:06.0 Off |                    0 |
| N/A   34C    P8             12W /   72W |       0MiB /  23034MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

Here is what each part tells you:

Field	What it tells you
Driver Version	The installed NVIDIA datacenter driver.
CUDA Version	The maximum CUDA the driver supports. Your workload’s CUDA build must be at or below this.
Name	The detected GPU model; confirm it matches the card you ordered.
Memory-Usage	Idle memory should read near 0 MB on a fresh node.
GPU-Util	Should read 0% on idle.
Persistence-M	On keeps the driver loaded between jobs, which suits long-running work.

If nvidia-smi is not found, or reports it cannot communicate with the driver, see Troubleshooting GPU Node. Once the card checks out, see Run GPU workloads in Docker for the recommended way to run a real job, or Save and reuse a GPU image to skip this setup on every new node.

(Optional) Confirm the CUDA toolkit

If you compile CUDA code directly on the host rather than inside a container, confirm the toolkit separately from the driver:

nvcc --version

On container-based images, nvcc may exist only inside the container, not on the host; that is expected, since the host only needs the driver.