---
title: "Model Endpoints"
description: "A TIR Model Endpoint serves your model on one or more workers behind one HTTPS address that you call with an auth token, with scaling, logs, and monitoring."
content_type: "overview"
product: "model-endpoints"
platform: "TIR"
url: "/tir/model-endpoints/"
---

# Model Endpoints

A Model Endpoint serves your model on one or more workers behind one HTTPS address, the root endpoint. Your applications call that address with an auth token. vLLM and SGLang endpoints answer the same chat and completion requests as the OpenAI API. Code written for OpenAI works once you change the address, the key, and the model name.

You choose a framework such as vLLM and tell the endpoint where the model comes from: Hugging Face, a [Model Repository](/tir/model-repository/) that holds your own files, or your own container image. Then you pick a machine and set how many workers run. The endpoint can keep a fixed number of workers, grow and shrink with traffic, or scale to zero when nobody calls it. An endpoint belongs to one project and one region.

## Where to find it

In TIR, open your project. In the side menu, under **Inference**, select **Model Endpoints**. Your endpoints in the current project and region are listed there, and **Create Endpoint** starts a new one.

## Create an endpoint

- [Create an endpoint](/tir/model-endpoints/how-to/create-an-endpoint/) from **Choose Framework** and one form.
- [Choose a framework](/tir/model-endpoints/how-to/choose-a-framework/) to match the model you bring.
- [Deploy a model from Hugging Face](/tir/model-endpoints/how-to/deploy-a-model-from-hugging-face/) with a model ID and a Hugging Face token.
- [Deploy a model from Model Repository](/tir/model-endpoints/how-to/deploy-a-model-from-model-repository/), including a LoRA adapter.
- [Deploy a custom container](/tir/model-endpoints/how-to/deploy-a-custom-container/) from a public or private image.
- [Set environment variables](/tir/model-endpoints/how-to/set-environment-variables/) for your model or container.
- [Choose a committed plan](/tir/model-endpoints/how-to/choose-a-committed-plan/) for steady, long-running use.

## Call an endpoint

- [Call an endpoint with a token](/tir/model-endpoints/how-to/call-an-endpoint-with-a-token/) from cURL, your own code, or a WebSocket client.
- [Use the OpenAI client and stream responses](/tir/model-endpoints/how-to/use-the-openai-client-and-stream-responses/).
- [Set up async invocation](/tir/model-endpoints/how-to/set-up-async-invocation/) to queue long-running requests.
- [Test in the Playground](/tir/model-endpoints/how-to/test-in-the-playground/) to chat with the model in the portal.

## Scale and operate

- [Set workers and autoscaling](/tir/model-endpoints/how-to/set-workers-and-autoscaling/), including scale to zero.
- [View logs and events](/tir/model-endpoints/how-to/view-logs-and-events/) to see what a worker is doing while it starts.
- [Monitor an endpoint](/tir/model-endpoints/how-to/monitor-an-endpoint/) with hardware and serving charts.
- [Update an endpoint](/tir/model-endpoints/how-to/update-an-endpoint/) to change its model, framework, machine, or settings.
- [Stop, start, and restart an endpoint](/tir/model-endpoints/how-to/stop-start-and-restart-an-endpoint/).
- [Clone an endpoint](/tir/model-endpoints/how-to/clone-an-endpoint/) to start a new one from the same settings.
- [Use a private network and a security group](/tir/model-endpoints/how-to/use-a-private-network-and-a-security-group/) to call the endpoint from your VPC.
- [Delete an endpoint](/tir/model-endpoints/how-to/delete-an-endpoint/) when you have finished with it.

## Walk through a model or framework

Each of these takes one model or framework from an empty project to a working request.

- On vLLM: [Llama 3 8B Instruct](/tir/model-endpoints/how-to/deploy-llama-3-8b-instruct-with-vllm/), [Llama 2](/tir/model-endpoints/how-to/deploy-llama-2-with-vllm/), [Code Llama 7B](/tir/model-endpoints/how-to/deploy-code-llama-7b-with-vllm/), [Gemma](/tir/model-endpoints/how-to/deploy-gemma-with-vllm/), and [MPT-7B-Chat](/tir/model-endpoints/how-to/deploy-mpt-7b-chat-with-vllm/).
- Applications on a vLLM endpoint: [Generate SQL from plain English with Code Llama](/tir/model-endpoints/how-to/generate-sql-from-plain-english-with-code-llama/), [Call a vLLM endpoint with the OpenAI client](/tir/model-endpoints/how-to/call-a-vllm-endpoint-with-the-openai-client/), and [Build an agent with the OpenAI Agents SDK](/tir/model-endpoints/how-to/build-an-agent-with-the-openai-agents-sdk/).
- Other frameworks: [Serve a model with NVIDIA Triton](/tir/model-endpoints/how-to/serve-a-model-with-nvidia-triton/), [Serve a PyTorch model with TorchServe](/tir/model-endpoints/how-to/serve-a-model-with-pytorch/), and [Deploy a custom API handler on a Custom Container](/tir/model-endpoints/how-to/deploy-a-custom-api-handler-on-a-custom-container/).

## Where to start

Start with [Get started with Model Endpoints](/tir/model-endpoints/get-started/), which deploys a Hugging Face model on vLLM and calls it. To create and manage endpoints from code, the TIR API reference lists the operations under [Model Endpoints](/api/tir/models/model-endpoints/). The [Model Endpoints FAQ](/tir/model-endpoints/faq/) answers questions about regions, access, scaling, and billing, and [Troubleshooting Model Endpoints](/tir/model-endpoints/troubleshooting/model-endpoints/) covers the messages that come up most.