Launch services on a private cluster

Run an instance, a virtual machine, a model endpoint, a training cluster, a Slurm cluster, or a vector database on the nodes your project has on a private cluster. You choose the cluster on that service’s own create form. An instance, model endpoint, training cluster, Slurm cluster, or vector database on the cluster is covered by what you already pay for the cluster’s nodes.

Before you begin

You need nodes allocated to your project on the cluster, with at least one of them showing Allocated. See Allocate nodes to projects.

Launch a service on the cluster

1. In the project that has the nodes, start creating the service as usual, for example an instance or a model endpoint.
2. Where the form asks for the compute resource, choose Private Cluster, then your cluster.
3. On a cluster with MIG on, choose a GPU instance size, such as MIG-1g.10gb. See GPU partitioning with MIG.
4. (Optional) To keep the service on, or off, a chosen set of nodes, set a Scheduling Rule under Affinity Group. See Use node affinity.
5. Finish the rest of the form, which that service’s own page explains, and create the service.

The service starts on one of your project’s nodes, and that node shows Occupied on Cluster Nodes. Projects that share a cluster can run different kinds of service at the same time.

Check what is running

On Project Allocations, open your project’s row to see its Deployed Services, each with the GPU cards it uses and its status. Filter the list to All Services, Instances, Model Endpoints, Training Cluster, Vector Database, or Slurm Clusters.

To look at one node, open it from Cluster Nodes. Its Deployed Services tab lists what runs on that node alone, and Monitoring shows its usage. When local storage is enabled on the node, a Local Storage tab shows the storage’s name, its size in GB, and the service attached to it.

Your service now runs on capacity your account owns. To track the nodes’ health over time, see Monitor a private cluster. If a service does not start on an allocated node, see Troubleshooting TIR Private Cluster.