Create a backend group

Create a backend group to name the servers that a Load Balancer sends traffic to: VMs, inference endpoints, or an Auto Scaling Group. A group has its own port, protocol, health check, and balancing method, and rules then forward requests to it. A second group lets different paths, or a small share of traffic for a new version, go to different machines.

Before you begin

• A Load Balancer. See Create a Load Balancer.
• Backends that are running in the same project: VMs, inference endpoints, or an Auto Scaling Group.
• Permission to update Load Balancers in the project. See TIR Access Control.

A backend group belongs to a single Load Balancer. Its name is required and must be unique in that Load Balancer.

Open the form

Open the Load Balancer, select the Backend groups tab, and select New group. You can also set up groups while you create the Load Balancer; see Create a Load Balancer.

Fill in the group

1. In Name, enter a name for the group.
2. In Protocol, choose how the Load Balancer talks to the backends: HTTP, HTTPS, or gRPC. HTTP is the default.
3. In Port, enter the port that the backends listen on, a whole number from 1 to 65535. It starts at 80. Any port is accepted, though HTTP usually uses 80 and HTTPS 443.
4. Choose the backends on one of the three tabs. A group holds one kind of backend, so clear one kind before you pick another.
   • Virtual Machines: tick the project’s ready VMs to add, and give each a Port and a Weight. TIR uses the VM’s public IP when it has one and its private IP otherwise. A VM marked “Cannot be used as a target” cannot be added.
   • Inference: tick the project’s running inference endpoints, each with a Port.
   • Auto Scaling Group: pick one of the project’s running Auto Scaling Groups. TIR attaches the group itself rather than a copy of its VMs. It is one backend, with a fixed port of 80 and a fixed weight of 1.
5. In Distribute requests by, choose how requests are shared:
   • Round Robin, the default, shares requests evenly across all backends.
   • Least Connection sends each request to the backend with the fewest open connections.
   • Source IP Hash sends the same client IP address to the same backend.
6. Set the Health check. It is on by default. It probes the backends and stops sending traffic to unhealthy ones. The fields are:
   • Probe protocol: TCP connects to the backend port, HTTP requests a path and checks the status code, and gRPC calls the standard gRPC health service. A group on HTTP or HTTPS offers TCP and HTTP, and starts with HTTP. A gRPC group offers TCP and gRPC.
   • Path and Success codes apply to the HTTP probe. The path starts as / and the success code as 200. Success codes take a comma-separated list such as 200, 204, and any other code fails the probe.
   • Interval (s) starts at 10 seconds and Timeout (s) at 3 seconds. Keep the timeout below the interval.
   • Healthy threshold is how many successful probes in a row an unhealthy backend needs before it gets traffic again. It starts at 2. Unhealthy threshold is how many failed probes in a row take a backend out of rotation. It starts at 3.
   • Each of these numbers is at least 1.
7. (Optional) Set Retry attempts, how many times a failed request is sent again to a different backend in the group. It starts at 0. Request timeout (s) starts at 300.
8. Select Create group.

A group needs a name, a valid port, and at least one backend. A backend and port can appear once in a group, and the same backend and port can also be in other groups. If you list one twice, select Remove duplicates; the group cannot be saved until the duplicate is gone.

Change or delete a group

In the Actions menu of the group, choose Edit or Delete. Saving an edit replaces the group’s list of backends as a whole, so a backend you leave out is removed. If a rule uses the group, the change reaches the running Load Balancer. TIR refuses a protocol change that would mix protocols inside a rule that uses the group.

Deleting removes the group and its backends from this Load Balancer. To delete a group that a rule still uses, remove it from the rule first.

A group gets traffic only once a rule forwards to it. To set that up, add a rule. For help with a message, see Troubleshooting TIR Load Balancer.