---
title: "Clone a cluster"
description: "Start a new TIR Slurm Cluster create form pre-filled from an existing cluster, including a Failed or Terminated one, and replace a failed cluster."
content_type: "how-to"
product: "tir-slurm-cluster"
platform: "TIR"
url: "/tir/tir-slurm-cluster/how-to/clone-a-cluster/"
topic: ["manage-the-cluster"]
---

# Clone a cluster

<!-- cspell:ignore sacct nodename RDMA sqsh nvcr DCGM sstat sawtoothing oauthtoken jobname FSDP -->

Open a new create form already filled in from an existing TIR Slurm Cluster. Clone a cluster to
repeat a setup, or to replace a cluster that failed. **Clone Cluster** is available at any status,
including **Failed** and **Terminated**.

## Before you begin

You need a cluster to copy. You also need Create permission on Training Clusters
in the project, because the action leads to the create form. Cloning fills in the form and stops
there, so the cluster is created, and billed, only once you submit it. The form follows the source cluster's
current settings, including any changes made since it was created.

## Clone the cluster

1. Select **Actions**, then **Clone Cluster**. The create form opens, filled in from the cluster.
2. Review every section, change what you want, and select **Create Training Cluster**.

The new cluster gets the original's image and version, SSH keys, security group, node count, and plan, and its TIR Private Cluster if it used one. It also gets the lifecycle script, the storage mounts with their mount paths, and the extra `slurm.conf` text with all four prolog and epilog scripts. A new cluster
name is generated for you.

Partitions stay behind, because a partition names the original cluster's own nodes. Live state
such as jobs, node health, and attached IPs stays behind too. Add your partitions again once the new
cluster is **Running**. See
[Manage partitions](/tir/tir-slurm-cluster/how-to/manage-partitions/).

## Replace a failed cluster

When a cluster ends up **Failed**, its reserved nodes are released and its security group and
storage selections are cleared. The failed record stays in the list, free of compute charges, until
you delete it. A Failed cluster stays Failed, so to get a **Running** cluster, replace it:

1. Clone the failed cluster. Check the security group and storage sections, which a failed cluster
   has cleared.
2. Change anything that might have caused the failure. The most common cause is a plan whose nodes are
   all taken, so try a different GPU type, fewer nodes, or a TIR Private Cluster.
3. Create the new cluster and wait until it reaches **Running**.
4. Delete the failed record. See
   [Terminate or delete a cluster](/tir/tir-slurm-cluster/how-to/terminate-or-delete-a-cluster/).

## Keep a setup for later

A cluster's configuration survives termination. You can clone a terminated cluster until you delete
it, so clone it first if you might want the same setup again. Terminating stops compute
billing, which makes terminating and cloning later cheaper than keeping a cluster idle. See
[Billing and plans](/tir/tir-slurm-cluster/concepts/billing-and-plans/).

To create a cluster from scratch instead, see
[Create a Slurm Cluster](/tir/tir-slurm-cluster/how-to/create-a-slurm-cluster/). If a create keeps
failing, see
[Troubleshooting TIR Slurm Cluster](/tir/tir-slurm-cluster/troubleshooting/tir-slurm-cluster/).