---
title: "Build an agent with the OpenAI Agents SDK"
description: "Deploy an instruct model on vLLM with tool calling turned on, then point the OpenAI Agents SDK at the endpoint so your agents use your own model."
content_type: "how-to"
product: "model-endpoints"
platform: "TIR"
url: "/tir/model-endpoints/how-to/build-an-agent-with-the-openai-agents-sdk/"
topic: ["tutorials"]
---

# Build an agent with the OpenAI Agents SDK

<!-- cspell:ignore Pydantic asyncio -->

Run the OpenAI Agents SDK against a model on your own Model Endpoint. The SDK lets an agent call tools, such as a function that looks up a seat or searches the web. Your endpoint runs the model on vLLM, so the agent works with open-weight models such as Llama or Mistral instead of a hosted model.

## Before you begin

- Permission to create model endpoints in the project. A Member needs a policy that grants create permission for the Inference service. See [Create an endpoint](/tir/model-endpoints/how-to/create-an-endpoint/).
- A Hugging Face access token with **Read** scope, saved as a Hugging Face integration. If the model is gated, the token's account also needs the owner's approval on the model's Hugging Face page. See [Create a Hugging Face integration](/tir/integrations/how-to/create-a-hugging-face-integration/).
- Python 3 on your computer or in an instance, and the SDK:

  ```bash
  pip install openai-agents
  ```

## Create the endpoint with tool calling

Create a vLLM endpoint that downloads the model from Hugging Face, with these values. Leave every other field as the form fills it. For what each field does, see [Deploy a model from Hugging Face](/tir/model-endpoints/how-to/deploy-a-model-from-hugging-face/).

| Field                             | Value                                                                                                                       |
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| Framework on **Choose Framework** | **vLLM**                                                                                                                    |
| **Download Type**                 | **Download from Huggingface**                                                                                               |
| **HuggingFace Model**             | An instruct model that supports tool calling, for example `meta-llama/Llama-3.3-70B-Instruct`                               |
| **HF Token Integration**          | Your Hugging Face integration                                                                                               |
| **Model Cache Storage**           | **Local drive**, with a size larger than the model's files                                                                  |
| **Select Computing Resource**     | **GPU** tab, a card and size with enough GPU memory for the model; a 70B model needs about 140 GB, spread over several GPUs |

Then turn on tool calling, which an agent needs to use its tools:

1. Open **Advanced Settings**. Under **LLM Settings**, in **System Settings**, turn **Enable Auto Tool Choice** on.
2. In **Tool Call Parser**, pick the parser that matches your model's family, such as **llama3_json** for Llama 3 models, **hermes** for Qwen models, or **mistral** for Mistral models. The list comes with the framework version you chose.
3. Select **Deploy Endpoint**.

Wait until the state is **Running**. A large model can take well over 15 minutes to download and load. With tool calling off, the endpoint still answers chat requests, but it ignores the agent's tools.

## Get the address and the token

On the endpoint's details page, select **API Request**. In **Sample API Request**, choose a token under **Auth token**, or select **Generate Token** to make one, and copy the **Root Endpoint**. Set both as variables in your terminal, where `<root-endpoint>` is the address you copied and `<auth-token>` is the token:

```bash
export ROOT_ENDPOINT="<root-endpoint>"
export AUTH_TOKEN="<auth-token>"
```

## Connect the SDK to the endpoint

Create a model provider that sends the SDK's requests to your endpoint. The base_url is the root endpoint followed by `/v1/`, and MODEL_NAME is the Hugging Face ID you entered in the form.

```python
import os
from openai import AsyncOpenAI
from agents import (
    Model,
    ModelProvider,
    OpenAIChatCompletionsModel,
    RunConfig,
    set_tracing_disabled,
)

BASE_URL = os.environ["ROOT_ENDPOINT"] + "/v1/"
API_KEY = os.environ["AUTH_TOKEN"]
MODEL_NAME = "meta-llama/Llama-3.3-70B-Instruct"

client = AsyncOpenAI(base_url=BASE_URL, api_key=API_KEY)
set_tracing_disabled(disabled=True)


class CustomModelProvider(ModelProvider):
    def get_model(self, model_name: str | None) -> Model:
        return OpenAIChatCompletionsModel(
            model=model_name or MODEL_NAME, openai_client=client
        )


CUSTOM_MODEL_PROVIDER = CustomModelProvider()
```

The code turns tracing off because the SDK would otherwise try to send traces to OpenAI.

## Try the SDK's example agents

The `openai-agents-python` repository has larger examples that you can point at your endpoint:

- A customer service agent: several agents that book seats, answer airline questions, and hand a customer to one another. Its tools are faq_lookup_tool and update_seat. Run it with python customer_service_agent.py.
- A research bot: it plans web searches, runs them in parallel, and writes a report.

Every tool an agent uses is registered with @function_tool. To run the research bot:

1. Clone the repository and open it:

   ```bash
   git clone https://github.com/openai/openai-agents-python.git
   cd openai-agents-python
   ```

2. In each example file, remove the lines that set a model, such as `model="gpt-4o"` or `model="o3-mini"`.
3. Add the code from **Connect the SDK to the endpoint** to the example file, and pass the run configuration to the run. In the research bot's manager.py, for example:

   ```python
   result = await Runner.run(
       planner_agent,
       f"Query: {query}",
       run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER),
   )
   ```

4. Run the example with python examples/research_bot/manager.py.

## Check that the agent works

The example prints the agent's answer. If an agent answers without calling a tool, check two settings in **LLM Settings**: **Enable Auto Tool Choice** must be on, and the **Tool Call Parser** must match the model. The settings an endpoint runs with show under **LLM Settings** on its **Overview** tab.

In production, use `Pydantic` models for structured input and output, keep tracing off to reduce logs, and use RunContextWrapper to keep context between tool calls.

A 70B model keeps several GPUs busy for as long as the endpoint runs. Between experiments, [stop the endpoint](/tir/model-endpoints/how-to/stop-start-and-restart-an-endpoint/) so that its hourly charges stop too.