Manage node pools

Worker node pools hold the worker nodes your pods run on. Each pool has one plan and one scaling strategy, so add a pool when you need a different kind of node, and resize a pool when you need more or fewer of the same. A cluster has one to 10 pools.

Before you begin

• Your cluster reads Running. The Node Pool tab shows its pools while the cluster is Running.
• On a prepaid account, you have infra credits for 24 hours of each hourly node you add, and the full price of each committed node.
• If you are a team member, your role has the update permission for Kubernetes, and the delete permission to delete a pool or a node.
• For labels, taints, and draining a node, kubectl is connected to the cluster. See Connect to a cluster with kubectl.

Add a pool

1. In MyAccount, under Compute, click Kubernetes. In the cluster’s row actions, click Add Node Pool, or open the cluster and click Add Node Pool there.
2. Fill in the drawer as on the create form: Node Pool Name, Scaling Strategy, and, under Infrastructure Configuration, Compute Family, Plan, and Billing Plan. Each choice is explained in Create a cluster. A pool holds 2 to 25 worker nodes, or 1 to 25 on the GPU or GDC 3rd generation family.
3. Click Add Pool.

The pool appears with New node pool added successfully. While it has no nodes yet, it shows Node pool is deploying. It will be available within 4-7 minutes.

Choose a pool action

While a pool reads Running, these actions are available on it:

• Edit Node Pool, for an autoscaling pool; see Configure autoscaling for a node pool.
• Resize Pool, for a pool on hourly billing.
• Convert to Committed or Committed Node Pool Settings, for a static pool, also while it reads Warning; see Use committed billing for a cluster.
• Delete Pool, in any state.

Resize a pool

Resizing works on a pool on hourly billing; a committed pool keeps its size for the term.

1. On the pool’s panel, click Resize Pool. Resize Node Pool opens.
2. Set the new worker count. In the portal, you can resize a pool to 2 to 24 worker nodes. An autoscaling pool stays within its minimum and maximum.
3. If you raise an autoscaling pool above its current size, choose what happens to its minimum:
   • Also update minimum desired nodes to raises the minimum to the new size once the scaling finishes, so autoscaling keeps the pool at least that big.
   • Keep minimum at leaves the minimum as it is, so autoscaling can remove the extra nodes again when they are not needed.
4. Confirm the resize.

Cardinality updated successfully. confirms the resize. If E2E has no GPU capacity when you grow a GPU pool, you see Currently inventory is not available; try again later.

Act on a worker node

To rename a worker node in the portal, edit its Name: 1 to 128 characters, starting with a letter and ending with a letter or digit, with only letters, digits, underscores, and hyphens.

A worker node’s Actions are:

• Start, for a node that reads Powered off, while the pool is not in Cooldown.
• Power Off.
• Reboot and Delete, for a node that is not Powered off.

No action is available while the node reads Powering on, Deploying, or Terminating.

Delete a worker node

A pool keeps at least 2 nodes on a CPU pool, and 1 on any other pool, such as GPU or GDC 3rd generation. At the minimum, a delete is refused with Cannot delete the node! The minimum node count for the pool is 2. On a pool whose minimum is 1, the message names 1. To go lower, delete the pool.

Before you delete a node, move its pods to other nodes. Use the node’s Kubernetes Worker Name from the Node Pool tab as <node-name>:

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

Delete the node from the Node Pool tab. kubectl delete node removes it from Kubernetes only; the machine keeps running and is billed.

Deleting a node takes capacity away from its pool and deletes anything stored on the node itself.

1. In the node’s Actions, click Delete. For a node in a committed pool, the dialog also warns that the node is reserved until its end date.
2. Confirm.

You see Node deletion process initiated successfully. It takes 3-5 minutes for the pool to get updated. For a node or pool on a committed plan, an early delete keeps the committed term’s charge.

Set labels and taints with kubectl

Set labels, taints, and other node settings with kubectl, using the node’s Kubernetes Worker Name as <node-name>:

kubectl label nodes <node-name> tier=frontend
kubectl taint nodes <node-name> dedicated=gpu:NoSchedule

The first command adds the label tier=frontend, which a pod can select with a node selector. The second keeps pods off the node unless they tolerate dedicated=gpu. To remove them, add a hyphen after the key:

kubectl label nodes <node-name> tier-
kubectl taint nodes <node-name> dedicated=gpu:NoSchedule-

A label or taint you set with kubectl is on that node only. A node that joins the pool later, such as one that autoscaling adds, starts without it.

To place pods on one pool, use the label E2E puts on every worker node, which holds the name of the node’s pool. To see it on each node, run:

kubectl get nodes -L worker_type

Then select the pool in the pod’s spec, where <pool-name> is the pool’s name on the Node Pool tab:

spec:
  nodeSelector:
    worker_type: <pool-name>

A node that a resize or scaling adds gets this label within about two minutes of joining the pool.

Delete a pool

A cluster keeps at least one pool. To replace its only pool, add the new pool first, then delete the old one.

Deleting a pool removes its worker nodes. Pods on them stop, and Kubernetes starts the ones a Deployment manages again on other pools that have room.

1. On the pool’s panel, click Delete Pool. For a committed pool, the dialog warns that the pool is reserved until its end date.
2. Under Type the exact node pool name, type the pool’s name, and confirm.

Deleting the last pool is refused with Cannot delete node service as minimum one node pool is required to run kubernetes cluster.

With pools sized for your applications, the cluster has the capacity you planned. For a pool that grows and shrinks on its own, see Configure autoscaling for a node pool. If a node reads NotReady or an action is refused, see Troubleshooting Kubernetes.