E2E CloudThe cluster details page
Everything you do with a cluster after you create it happens on its details page, which has 10 tabs and an Actions menu. Find the control you are looking for here, then follow the link to the page that covers the task.
Open a cluster
Section titled “Open a cluster”In TIR, open your project. In the side menu, under PRODUCTS, select Training Cluster, then select the cluster’s name in Manage Training Clusters. A cluster belongs to one project, so first check that the project selector at the top of the console shows that project. To create a cluster, see Create a Slurm Cluster.
The page header
Section titled “The page header”The top of the page has the cluster’s status, Connect, and the Actions menu. To move to another cluster, use Select Training Cluster. While the cluster is Running, Restart Workers there does the same as Restart All Workers. For what each status allows, see Cluster statuses and actions.
Connect gives ready-made commands to copy: an SSH command for each address the cluster has,
then sinfo, scontrol show nodes, sbatch, sstat, and scontrol show job. You can also open it
from the cluster’s row in the list once the cluster reaches Initializing Worker Nodes. See Connect to a
cluster.
The tabs
Section titled “The tabs”The details page has 10 tabs, in this order:
| Tab | What it is for | Page |
|---|---|---|
| Details | The cluster’s name, image version, status, node count, plan, and how to connect. | The Details tab |
| Cluster Overview | A summary of GPUs and jobs, node health, and the Slurm partition table. | Check cluster and node health |
| Nodes | Per-node and per-GPU metrics, health filters, XID errors, and node reboot. | Check cluster and node health |
| Jobs | The live Slurm queue, with state counts. | See jobs and logs |
| Monitoring | GPU and scheduler charts over a time range you choose. | Monitor a cluster |
| Alerts | Attach alerts to the cluster. | Monitor a cluster |
| Logs | Logs from the login node and the worker nodes. | See jobs and logs |
| Volumes | Mount and unmount datasets, Shared File System, and Parallel File System volumes. | Mount and unmount volumes |
| Network & Security | The public IP, a VPC IP, and security groups. | Manage network and security |
| Login User Management | Individual SSH logins and groups. | Manage login users |
Details is always available, whatever the cluster’s status, and is where you check the status and plan of a Failed or Terminated cluster. Nodes opens as soon as the cluster reaches Initializing Worker Nodes. Every other tab opens when the cluster is Running.
The Details tab
Section titled “The Details tab”Details has the cluster’s image version, status and node count under Cluster Details, its plan and price under Plan Details, and the root SSH keys and SSH command under Connection Details.
The image version names what it holds: v2-slurm25.11-ubuntu24.04 means Slurm 25.11 on Ubuntu 24.04.
Directives and flags change between Slurm releases, so read the Slurm
documentation for the version you have.
Number of Nodes is the worker node count. To change it, use Scale Cluster. See Scale a cluster.
The Actions menu
Section titled “The Actions menu”Actions is on the details page and on each row of the list, with the same actions in both. You see only the actions your update permission allows.
| Action | What it does | Page |
|---|---|---|
| Update SSH Keys | Replaces the whole set of root SSH keys. | Update a cluster |
| Update Image | Changes the cluster’s own Slurm, driver, and operating system image, with a rolling restart. | Update a cluster |
| Slurm Configuration | Opens the dialog for extra slurm.conf directives, partitions, and prolog and epilog scripts. |
Change the Slurm configuration |
| Scale Cluster | Adds or removes worker nodes. | Scale a cluster |
| Convert to Committed | Moves an hourly cluster to a committed plan. Only the billing changes. | Convert a cluster to committed |
| Restart All Workers | Restarts every worker node. | Update a cluster |
| Restart Cluster | Restarts the controller, the login node, every worker, and the accounting database. | Update a cluster |
| Restart Login Service | Restarts the login node. | Update a cluster |
| Reconfigure Cluster | Re-applies the cluster’s stored settings exactly as they are. | Update a cluster |
| Clone Cluster | Starts a create form pre-filled from this cluster. The cluster is created only when you submit it. | Clone a cluster |
| Terminate Cluster | Releases the nodes and ends the cluster. The record stays as Terminated. | Terminate or delete a cluster |
| Delete Training Cluster | Removes the cluster from the list. The details page labels it Delete Cluster. | Terminate or delete a cluster |
The actions you see depend on the cluster’s status and plan. Cluster statuses and actions lists them by status and says what each one does to running jobs.
Where is that control
Section titled “Where is that control”| I am looking for | It is at |
|---|---|
| The SSH command | Details, then Connection Details, or the Connect button |
| The cluster’s IP | The same places, or Network & Security, then Network |
| The image version | Details, then Cluster Details |
| A partition | Cluster Overview, then the Slurm partition table |
The extra slurm.conf settings |
Actions, then Slurm Configuration, then the slurm.conf tab |
| A prolog or epilog script | Actions, then Slurm Configuration, then the Prolog & Epilog tab |
| Why a job is pending | Jobs for its state, then squeue -o "%R" over SSH for the reason |
| A broken GPU | Nodes, then the XID Errors filter |
| Reboot one node | Nodes, then View GPUs on the node, then Reboot Node |
| More nodes | Actions, then Scale Cluster |
| Mount a volume | Volumes, then the volume’s type, then Mount |
| Add a teammate | Login User Management, then Members, then Create Member |
| Open a port | Network & Security, then Security Groups |
| Attach an alert | Alerts |
| Stop paying for the cluster | Actions, then Terminate Cluster |
| Reuse a cluster’s configuration | Actions, then Clone Cluster |
What the markers mean
Section titled “What the markers mean”Node states and health badges are explained where you read them, in Check cluster and node health. The other markers across the cluster’s pages are:
- Floating on an IP means it is assigned from a pool and can change. Reserved means it is held for your account and stays the same.
- An asterisk beside a partition name marks the default partition.
- Managed on a partition means E2E manages it, so Edit and Delete appear only on your own partitions.
- “not live” on a partition means a partition you configured is missing from what the controller reports.
- DISABLED marks one of your own partitions that has Enabled switched off.
- Latest and Current on an image version mean the newest available and the one running now.
- Inventory not available on a plan in the create form means every node of that type is taken for now.