Update an endpoint

Change an endpoint’s framework, model, machine, or settings and keep using the same endpoint.

Before you begin

• Permission to update model endpoints in the project. A Member needs a policy that grants update permission for the Inference service. Update Inference is available with the update permission.
• Credit and free capacity if you change the machine. TIR checks them again when you update.

Update the endpoint

1. In Model Endpoints, find the endpoint. Under Actions, or in the Actions menu on its details page, select Update Inference.
2. Change what you need: the framework and its version, the model, the machine, or Advanced Settings. The endpoint’s current LLM Settings are listed on its Overview.
3. Select Update Endpoint. On a committed endpoint, confirm the committed plan again. Your saved end-of-term choice is already selected, and you can change it there.

Change the worker counts on the Serverless Configuration tab instead. See Set workers and autoscaling. A committed endpoint’s plan cannot be changed here.

To change only a Custom Container’s image, ports, or commands, use Update Container Configuration. See Deploy a custom container.

What happens to the endpoint

• On a Running endpoint, or one in any state except Stopped, the change applies to the running endpoint.
• On a Stopped endpoint, the change is saved and the endpoint stays stopped.
• If you change the plan to another kind of machine, for example from GPU to CPU, TIR re-creates the endpoint’s workers. The endpoint shows Waiting until they are up.

Check the update

TIR opens the endpoint’s details. Watch the state there until it is Running. If the update fails, TIR reports “Unable to update endpoint” or the reason. TIR can also refuse an update for the moment with “action not allowed” or “Error while updating the inference”. This can happen when the endpoint changed state in the meantime. Try again. See Troubleshooting Model Endpoints.