---
title: "Get started with Model Endpoints"
description: "Deploy a Hugging Face model on vLLM as a Model Endpoint, wait for it to reach Running, and send it a first request with an auth token."
content_type: "get-started"
product: "model-endpoints"
platform: "TIR"
url: "/tir/model-endpoints/get-started/"
---

# Get started with Model Endpoints

Deploy a chat model from Hugging Face on vLLM, wait for it to start, and send it a first request. You need a TIR project and permission to create model endpoints in it. A Member needs a policy that grants create permission for the Inference service.

You also need:

- A Hugging Face access token saved as a Hugging Face integration. See [Create a Hugging Face integration](/tir/integrations/how-to/create-a-hugging-face-integration/).
- The ID of a chat or instruct model on Hugging Face, in the form `<organization>/<model-name>`. If the model is gated, accept its licence on Hugging Face with the account that owns the token.

## Create the endpoint

1. In TIR, open your project. In the side menu, under **Inference**, select **Model Endpoints**, then **Create Endpoint**.
2. Find **vLLM**, keep its suggested **Version**, and select **vLLM**.
3. Keep the suggested **Endpoint Name**, which starts with `vllm-`, or type your own in lowercase letters, digits, and hyphens.
4. Leave **Download Type** on **Download from Huggingface**. In **HuggingFace Model**, type the model ID, and wait for TIR to accept it.
5. In **HF Token Integration**, choose your integration. TIR checks that the token can read the model.
6. Leave **Model Cache Storage** on **Local drive** with its 100 GB size, which holds a model of 7 to 8 billion parameters with room to spare.
7. In **Select Computing Resource**, open the **GPU** tab and choose a card with enough GPU memory for the model. In 16-bit precision, the weights take about 2 GB of GPU memory for every billion parameters, so an 8 billion parameter model needs about 16 GB plus room for requests. Pick a size under **Configuration**, and choose **On-Demand** under **Instance Pricing** to pay by the hour.
8. In **Serverless Configuration**, leave **Active Worker** at 1 and set **Max Workers** to 1, so the endpoint keeps one worker.
9. Select **Deploy Endpoint**.

TIR opens the endpoint's details. The endpoint starts in **Deploying** while its worker downloads and loads the model, which can take 10 to 15 minutes or more for a large model. When the state reads **Running**, the endpoint is ready for requests.

## Call the endpoint

1. On the **Overview** tab, under **Auth token**, choose a token, or select **Generate token** to create one.
2. Copy the endpoint address, the root endpoint.
3. Send a chat request. Add `/v1/chat/completions` to the root endpoint, put the token in the `Authorization` header, and use the model ID as the model name:

```bash
curl -X POST "<root-endpoint>/v1/chat/completions" \
  -H "Authorization: Bearer <auth-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<organization>/<model-name>",
    "messages": [{"role": "user", "content": "Who won the world series in 2020?"}]
  }'
```

Replace `<root-endpoint>` with the address you copied, `<auth-token>` with the token you chose, and `<organization>/<model-name>` with your model ID. **API Request** on the endpoint's page gives the same request with these values filled in.

## Check that it works

The endpoint shows **Running** with its workers counted under the state, for example 1 / 1. Your request returns a JSON reply, and the model's answer is in `choices[0].message.content`. If the endpoint stays in **Deploying** or the request is refused, see [Troubleshooting Model Endpoints](/tir/model-endpoints/troubleshooting/model-endpoints/).

## Stop the charges when you are done

An hourly endpoint is charged for the time its worker runs, including the time it spends loading the model, even when it sits idle. When you finish testing, select **Stop Service** on the endpoint's details page, or under **Actions** in the list. The endpoint keeps its settings and address, and **Start Service** brings it back. To remove it for good, see [Delete an endpoint](/tir/model-endpoints/how-to/delete-an-endpoint/).

## What to do next

- [Use the OpenAI client and stream responses](/tir/model-endpoints/how-to/use-the-openai-client-and-stream-responses/) to call the endpoint from Python.
- [Set workers and autoscaling](/tir/model-endpoints/how-to/set-workers-and-autoscaling/) to let the endpoint grow with traffic, or scale to zero so an idle endpoint costs nothing.
- [Test in the Playground](/tir/model-endpoints/how-to/test-in-the-playground/) to chat with the endpoint in the portal.
- Follow a walkthrough for a particular model, such as [Deploy Llama 3 8B Instruct with vLLM](/tir/model-endpoints/how-to/deploy-llama-3-8b-instruct-with-vllm/).