Build an agent with the OpenAI Agents SDK

Run the OpenAI Agents SDK against a model on your own Model Endpoint. The SDK lets an agent call tools, such as a function that looks up a seat or searches the web. Your endpoint runs the model on vLLM, so the agent works with open-weight models such as Llama or Mistral instead of a hosted model.

Before you begin

• Permission to create model endpoints in the project. A Member needs a policy that grants create permission for the Inference service. See Create an endpoint.

• A Hugging Face access token with Read scope, saved as a Hugging Face integration. If the model is gated, the token’s account also needs the owner’s approval on the model’s Hugging Face page. See Create a Hugging Face integration.

• Python 3 on your computer or in an instance, and the SDK:

  pip install openai-agents

Create the endpoint with tool calling

Create a vLLM endpoint that downloads the model from Hugging Face, with these values. Leave every other field as the form fills it. For what each field does, see Deploy a model from Hugging Face.

Field	Value
Framework on Choose Framework	vLLM
Download Type	Download from Huggingface
HuggingFace Model	An instruct model that supports tool calling, for example meta-llama/Llama-3.3-70B-Instruct
HF Token Integration	Your Hugging Face integration
Model Cache Storage	Local drive, with a size larger than the model’s files
Select Computing Resource	GPU tab, a card and size with enough GPU memory for the model; a 70B model needs about 140 GB, spread over several GPUs

Then turn on tool calling, which an agent needs to use its tools:

1. Open Advanced Settings. Under LLM Settings, in System Settings, turn Enable Auto Tool Choice on.
2. In Tool Call Parser, pick the parser that matches your model’s family, such as llama3_json for Llama 3 models, hermes for Qwen models, or mistral for Mistral models. The list comes with the framework version you chose.
3. Select Deploy Endpoint.

Wait until the state is Running. A large model can take well over 15 minutes to download and load. With tool calling off, the endpoint still answers chat requests, but it ignores the agent’s tools.

Get the address and the token

On the endpoint’s details page, select API Request. In Sample API Request, choose a token under Auth token, or select Generate Token to make one, and copy the Root Endpoint. Set both as variables in your terminal, where <root-endpoint> is the address you copied and <auth-token> is the token:

export ROOT_ENDPOINT="<root-endpoint>"
export AUTH_TOKEN="<auth-token>"

Connect the SDK to the endpoint

Create a model provider that sends the SDK’s requests to your endpoint. The base_url is the root endpoint followed by /v1/, and MODEL_NAME is the Hugging Face ID you entered in the form.

import os
from openai import AsyncOpenAI
from agents import (
    Model,
    ModelProvider,
    OpenAIChatCompletionsModel,
    RunConfig,
    set_tracing_disabled,
)

BASE_URL = os.environ["ROOT_ENDPOINT"] + "/v1/"
API_KEY = os.environ["AUTH_TOKEN"]
MODEL_NAME = "meta-llama/Llama-3.3-70B-Instruct"

client = AsyncOpenAI(base_url=BASE_URL, api_key=API_KEY)
set_tracing_disabled(disabled=True)


class CustomModelProvider(ModelProvider):
    def get_model(self, model_name: str | None) -> Model:
        return OpenAIChatCompletionsModel(
            model=model_name or MODEL_NAME, openai_client=client
        )


CUSTOM_MODEL_PROVIDER = CustomModelProvider()

The code turns tracing off because the SDK would otherwise try to send traces to OpenAI.

Try the SDK’s example agents

The openai-agents-python repository has larger examples that you can point at your endpoint:

• A customer service agent: several agents that book seats, answer airline questions, and hand a customer to one another. Its tools are faq_lookup_tool and update_seat. Run it with python customer_service_agent.py.
• A research bot: it plans web searches, runs them in parallel, and writes a report.

Every tool an agent uses is registered with @function_tool. To run the research bot:

1. Clone the repository and open it:

   git clone https://github.com/openai/openai-agents-python.git
   cd openai-agents-python

2. In each example file, remove the lines that set a model, such as model="gpt-4o" or model="o3-mini".

3. Add the code from Connect the SDK to the endpoint to the example file, and pass the run configuration to the run. In the research bot’s manager.py, for example:

   result = await Runner.run(
       planner_agent,
       f"Query: {query}",
       run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER),
   )

4. Run the example with python examples/research_bot/manager.py.

Check that the agent works

The example prints the agent’s answer. If an agent answers without calling a tool, check two settings in LLM Settings: Enable Auto Tool Choice must be on, and the Tool Call Parser must match the model. The settings an endpoint runs with show under LLM Settings on its Overview tab.

In production, use Pydantic models for structured input and output, keep tracing off to reduce logs, and use RunContextWrapper to keep context between tool calls.

A 70B model keeps several GPUs busy for as long as the endpoint runs. Between experiments, stop the endpoint so that its hourly charges stop too.