Save and reuse a GPU image

Getting a fresh GPU node ready by hand, with the framework packages and container images your job needs, commonly takes 15 to 25 minutes. A node launched from a saved image that already holds them is ready in about 3 to 6 minutes, so saving an image once pays off from the second node on. Save Image works the same way it does on any node (see Save a node as an image). On a GPU node, what matters is what to bake in, and how a few GPU settings behave in a saved image.

Snapshots depend on the node family: GDC3 nodes support snapshots, and on GDC nodes Save Image is the way to capture a re-launchable template of the root disk.

What to bake into a GPU image

Component	Why bake it
NVIDIA driver	Skips the kernel-module install and reboot on first boot.
CUDA toolkit, if you compile on the host	Skips a multi-GB toolkit download.
Docker and the NVIDIA Container Toolkit	Skips the nvidia-ctk runtime configure step on every new node.
Pulled container images	Skips a multi-GB image pull for every replica.
Conda or pip environments	Skips the PyTorch, JAX, vLLM, or TGI install path.
cuDNN and NCCL	Most frameworks need them; check they came with the toolkit.
Firewall and sysctl settings	Every node starts with the same tuning.

Keep large model weights and datasets on a volume and mount them, so the image stays small. Inject secrets, API keys, and anything per-node at runtime rather than baking them in.

Save the GPU node as an image

1. Set up everything in What to bake into a GPU image. Then check it end to end: run nvidia-smi, run a container with --gpus all, and run a single inference or training step.

2. Turn on persistence mode:

   sudo nvidia-smi -pm 1

   On Ubuntu 24.04 with the 580.x driver, it stays on after a reboot by itself. On driver 570.x and earlier, also run sudo systemctl enable nvidia-persistenced.

   See Troubleshooting GPU Node if that command, or enabling the persistence service, returns an error.

3. GDC3 nodes can run Save Image while Running; GDC nodes need to be Powered off first, the same rule as most other node families.

4. Run Save Image. GPU images capture the entire root disk, driver, toolkit, and container caches included, and can reach hundreds of GB; they are billed for their full size while retained, so delete a saved image you no longer need from the Manage Saved Images page (Nodes, then Saved Images). Saving an image of 50 to 300 GB takes about 10 to 20 minutes.

5. Wait for the image to read Ready on the Manage Saved Images page. To launch a new node from it, open the image’s Actions menu and click Create Node. The create-node flow opens with your image as the base; pick a plan in the same region with a root disk at least as big as the image.

A node launched from the image boots with the driver, toolkit, and pulled container images already in place. Run nvidia-smi on it once to confirm the card, as in Connect to a Linux GPU node. If the card is missing or the driver will not load, see Troubleshooting GPU Node.

Bake one image per card-family-and-framework combination, and re-bake it when you upgrade the framework or driver, rather than trying to make one image cover every workload.

Plan upgrade and reinstall

Plan Upgrade supports moving within the same card family, for example a 1x plan to a 2x or 4x plan on the same card, or a bigger vCPU/RAM SKU on that card. To move to a different card model, an L40S to an A100 for example, save an image and launch a new node on the target card. Plan changes go up only, on every plan including committed ones, so size up in steps you are sure of.

Reinstall wipes the root disk, including the driver and toolkit. If you do not have a saved image, plan for the full driver-and-framework install path again on first boot after a reinstall.