Use node affinity

Group a private cluster’s nodes, then keep a workload on that group’s nodes, or off them, when you launch it. By default, TIR can place a workload on any node in the cluster. With node groups you decide, for example, that inference runs only on the nodes set aside for it, or that experiments stay away from the nodes running production work.

Diagram as text

flowchart LR
    N[Private cluster nodes] --> G["Node group: inference-nodes"]
    G -->|Affinity| R1[Workload runs only on these nodes]
    N -->|Anti-Affinity on inference-nodes| R2[Workload runs on any other node in the cluster]
    N -->|All Nodes, or rule left unset| R3[Workload runs on any node in the cluster]

Before you begin

• A private cluster with nodes allocated to your project. See Allocate nodes to projects.
• Update access to every project whose nodes you want in the group. You need the same access to edit or delete the group later.

Plan your groups

Name a group for what it does, such as inference-nodes, gpu-nodes-1, worker-1, testing-workers, or high-workload-nodes, rather than by node ID. The name then stays meaningful as the cluster’s layout changes. Keep each group small and specific to the workloads that use it.

Then choose a rule for each workload:

Scheduling Rule	Where the workload runs	Use it for
All Nodes	Any node in the cluster, the same as leaving the rule unset	Workloads that can run on any node
Affinity	Only the group’s nodes	Work that needs particular hardware, such as GPU-specific or high-memory nodes
Anti-Affinity	Any node in the cluster except the group’s nodes	Experimental or bursty jobs you want kept off stable production nodes

Create a node group

1. Open the cluster and select the Node Affinity tab. The page is titled Node Groups, with the cluster’s Total Nodes, the number of Groups, and the groups you already have.
2. Select + New group.
3. Name the group, up to 64 characters. Each group on a cluster needs its own name.
4. Select one or more of the cluster’s nodes for it, and confirm.

The group appears on the page with its name, its number of nodes, and up to four of their names.

Apply a group to a workload

On the service’s create form, under the optional Affinity Group section, choose a Scheduling Rule. For Affinity or Anti-Affinity, also choose the node group. See Launch services on a private cluster.

Edit a node group

Before you edit a group, check which workloads use it, so that you know which ones a change affects.

1. On the Node Affinity tab, select Edit on the group.
2. Change the name, or the set of nodes. The nodes you select become the group’s complete new list.
3. Save the change.

An instance or model endpoint already running under the group’s rule stays where it is. To move it onto the updated group, remove the rule from it, update the group, then apply the rule again.

Delete a node group

1. Remove the group’s rule from every instance and model endpoint that uses it.
2. On the Node Affinity tab, select Delete on the group, and confirm.

While an instance or model endpoint still uses the group, TIR refuses the delete with “Private-cloud affinity is being used by notebook or inference services.”

Your workloads now land on the nodes you meant for them. If one does not start on the nodes you expect, see Troubleshooting TIR Private Cluster.