Create a Slurm Cluster
The create form is one page. This walks it top to bottom — every field, what it accepts, and the decisions worth thinking about before you click.
Open the Create Form
- Log in to the TIR console.
- Select your project in the top-left project selector.
- Click Training Cluster in the sidebar.
- Click Create Training Cluster.
The page is headed Training Clusters — "Create and manage your training clusters for distributed machine learning workloads."
The form says so plainly: "Charges apply from the moment the cluster is created." An idle cluster costs the same as a busy one. Do not create a cluster the day before you need it.
Cluster Name
| Required | Yes |
| Default | A generated name such as training-cluster-260910092046 |
| Rules | Lowercase letters, digits and hyphens; must start with a letter; must not end with a hyphen; at most 50 characters |
| Errors | Cluster name is required · Name is too long, max 50 characters allowed. · Invalid name format |
The name is how you and your team recognise the cluster in the list, and it appears in the cluster's
internal resource names. Pick something meaningful — llama-pretrain-sep, not the default.
Image
Two dropdowns under the label Select image and version, plus a refresh icon.
| Field | What it selects |
|---|---|
| Image | The image family, e.g. Ubuntu Slurm or NeMo Framework |
| Version | A specific build, e.g. v2-slurm25.11-ubuntu24.04 |
The version encodes the Slurm version and base OS, and is shown on the cluster's Details tab afterwards. Each image comes with GPU drivers, CUDA, NCCL, the Slurm runtime, and the Enroot and Pyxis container stack pre-installed.
| Family | Use it for |
|---|---|
| Ubuntu Slurm | The general-purpose choice. A clean Slurm environment; bring your framework in a container |
| NeMo Framework | NVIDIA NeMo / Megatron-style LLM training, with the framework stack already in the cluster image |
The Image field selects the cluster's own software — Slurm, the drivers, the OS. It does not
constrain what your jobs run in. To run a specific PyTorch, JAX or custom image, pass
--container-image to srun/sbatch; see
Containers with Enroot and Pyxis.
If you have been told to create a new cluster to get a different framework version, you almost certainly do not need to.
A container per job means two teams can use different CUDA and framework versions on one cluster, and upgrading your framework never means touching the cluster. Reach for NeMo Framework only when you specifically want that stack baked into the nodes.
The version can be changed later with Update Image, which does a rolling restart.
Plan Configuration
Tabs select where the capacity comes from: GPU (on-demand inventory) or Private Cluster (capacity you have already reserved). Some accounts also see an IndiaAI tab for IndiaAI-allocated capacity.
GPU tab
Plan cards show the GPU model and per-node resources, for example:
NVIDIA H200 SXM
8 GPU, 240 CPU, 3000 GB RAM, 141GB GPU Memory
₹3,484.00 /hr
A greyed-out card with Inventory not available means no free nodes of that type right now.
| Field | Notes |
|---|---|
| Configuration | The per-node shape, e.g. 8 GPU - 240 vCPUs - 3000GB RAM |
| Workers | Number of worker nodes, set with the − / + stepper. Minimum 1 |
Below that, Pricing offers On-Demand ("Flexible pay-as-you-go hourly pricing") and, where available, savings plans — 1 Month (Save 10%), 3 Month (Save 14%) and 6 Month (Save 20%).
The rate on the card is for one node. Two H200 nodes cost twice the card price. The savings plans commit you for their term.
Private Cluster tab
Select Private Cluster — "Choose a pre-existing private cluster. Click here to create new one."
Each card shows the cluster name, GPU model, and chips for Total Nodes and Available Nodes. Then
set Workers from the available nodes.
| Error | Meaning |
|---|---|
| The requested node count exceeds current availability. Please reduce the node count. | You asked for more nodes than the private cluster has free |
| Node count cannot be less than 1 | At least one worker is required |
A Private Cluster holds nodes for your account. Creation always succeeds, provisioning is faster, and there are no separate hourly charges for the Slurm cluster on top. This is the right pattern if you tear down and recreate clusters regularly.
- Committed and savings plans cannot be scaled. Size the cluster before you commit.
- Scaling down restarts the remaining workers, so it is not a free adjustment.
If you are unsure, start with fewer nodes on On-Demand and scale up once the queue tells you what you need.
Missing plan selection raises Please select a plan configuration.
Access
SSH Keys
| Required | Yes |
| Helper | "Choose a pre-existing SSH key for seamless access by uploading the key file. Click here to create a new one." |
| Error | Please select at least one SSH key |
Select one or more keys; Click here creates a new one inline. These keys give root access to the
login node.
A cluster with one key is one lost laptop away from being unreachable. Add a colleague's key or a backup key at create time. Keys can be changed later with Update SSH Keys, and per-person access is better handled with Login User Management.
Parallel File System
| Required | Yes — and at least one storage volume is required overall |
| Helper | "Select a parallel file system and specify the path where it will be mounted on the cluster workers. Click here to create new one." |
Select a PFS volume and give it a mount path, e.g. /pfs. Only volumes in a healthy state are
offered.
| Mount path rule | Detail |
|---|---|
Must start with / | pfs is invalid, /pfs is valid |
Cannot be / | The root itself is not allowed |
No //, no trailing / | /a//b and /pfs/ are invalid |
| No spaces | The space key is blocked in the field |
| Must be unique | Errors with All mount paths must be unique across your PFS, SFS and dataset paths |
| Errors | Mount path is required · Invalid path format |
The mount appears at the same path on the login node and every worker, so it is where datasets, checkpoints, logs, job scripts and container squash files belong. Anything outside a mounted volume is scratch. Choose a short, obvious path and keep it consistent across clusters.
Advanced Settings
Expand Advanced Settings for the rest. One field here is required.
Security Group
| Required | Yes |
| Helper | "Choose a pre-existing Security Group to manage network access. Click here to create a new one." |
| Error | Please select a Security Group |
Once you select SSH keys, the form auto-selects a security group that has SSH open, and groups
without port 22 are disabled with a Port 22 disabled chip. If SSH keys are selected but no
suitable group is, an inline note reads "Security Group with port 22 is required when SSH Key is
added".
Provisioning still succeeds — you simply cannot SSH in. Leave the auto-selected SSH group in place unless you have a specific reason, and tighten the source range rather than removing the rule. Groups can be changed later from Network & Security.
Lifecycle Script (Optional)
Lifecycle configuration script that you want to run on each instance in the instance group after cluster creation
Click Add Script to open a code editor, or Upload Script to load a .sh or .py file.
✕ Remove Script clears it. Uploading any other file type raises Please upload a valid .sh or .py
file.
Use it for one-time node setup — installing a package, writing a config file, mounting something extra.
The lifecycle script runs on each node after the cluster is created, and it is a create-time field you cannot change later. For code that must run around every job, use Worker Prolog instead — that is editable at any time. The two coexist; a prolog never replaces the lifecycle script.
It is stored with the cluster configuration and carried into clones. Read credentials from a file on your mounted storage at runtime instead.
Slurm Configuration (Optional)
Extra
slurm.confsettings and prolog/epilog scripts for the cluster. Partitions are added from the cluster's Overview tab once it is running, since a partition selects specific nodes.
Click Configure Slurm to open the dialog with two tabs — slurm.conf and Prolog & Epilog —
and confirm with Save Configuration. The summary next to the button reads Not configured, or a
list such as extra slurm.conf, 1 script. Clear Configuration discards it.
There is no Partitions tab here. Add partitions once the cluster is Running, from the partition table.
Everything in this dialog is editable on a Running cluster. Setting it at create time only saves you one reconfigure later. See Slurm Configuration.
Shared File System (Optional)
Select an SFS volume and a mount path (placeholder e.g. /shared). Read-write, and mounted on every
node. The same path rules as PFS apply.
Login User Management — per-user SSH logins and private home directories — provisions those home directories on an SFS volume, and its storage picker lists SFS only. Attaching one here costs nothing; attaching one later means a node restart that ends running jobs.
Dataset Storage (Optional)
Select a dataset and a mount path (placeholder e.g. /data).
Perfect for input data, useless for checkpoints and logs. Those need PFS, SFS or Weka. A job that tries to write to a dataset mount fails with a permission error.
Weka Storage
In regions where Weka is available it replaces PFS as the required volume, with the same shape and
path rules (placeholder e.g. /weka).
Create the Cluster
Click Create Training Cluster. The button stays disabled while any field is invalid.
Checks that run on submit:
| Check | Message |
|---|---|
| A plan is selected | Please select a plan configuration |
| At least one volume | Please select at least one storage (PFS, SFS, or Dataset) — or (Weka, SFS or Datasets) in Weka regions |
| Mount paths are unique | All mount paths must be unique |
| Partitions are not set at create | Partitions can only be configured after the cluster is running, because a partition selects specific nodes and those do not exist yet |
| Enough credit for the plan and node count | A credit message from the billing service |
After you submit
- The cluster appears in the list at status Creating. The list refreshes itself every few seconds while anything is Creating.
- Provisioning normally takes a few minutes: nodes are reserved, the Slurm cluster is installed, and the login endpoint is published.
- When every component is ready, the status becomes Running.
Try Reconfigure Cluster — it is available on a Creating cluster and re-applies the intended state. If the cluster ends up Failed, it cannot be repaired: use Clone Cluster to get the same configuration in a fresh form, then delete the failed record. A Failed cluster is not billed for compute.
Next
- Submit your first job — connect and run something.
- Add partitions — now that nodes exist.
- Cache a container image — before your first real run.
- Enable Login User Management — if a team will share the cluster.
Create via the API
Every field above maps to the create endpoint, so cluster provisioning can be scripted. See the
Training Cluster API reference
for the payload, including the optional slurm_config object.