---
title: "Monitor an endpoint"
description: "Read the hardware and serving charts of a Model Endpoint on the Monitoring tab, compare its workers, and attach alerts from the Settings tab."
content_type: "how-to"
product: "model-endpoints"
platform: "TIR"
url: "/tir/model-endpoints/how-to/monitor-an-endpoint/"
topic: ["operate"]
---

# Monitor an endpoint

<!-- cspell:ignore TFLOP -->

Follow an endpoint's load with charts for GPU, CPU, and memory. For vLLM, SGLang, and vLLM-Omni endpoints, charts also show the serving speed. Use them to decide when to change the workers or the machine.

## Before you begin

- Permission to read model endpoints in the project. A Member needs a policy that grants read permission for the Inference service.
- An endpoint that is serving. The **Monitoring** tab says "Metrics are not available for the current service status." while the endpoint is **Stopped**, **Terminating**, **Error**, or **Idle**.

## Open the charts

1. Open the endpoint and select the **Monitoring** tab.
2. Choose **Hardware** for GPU, CPU, and memory use, which every endpoint has. Choose **Service** for serving speed: it has charts for vLLM, SGLang, and vLLM-Omni endpoints, and for a Custom Container that runs a vLLM or SGLang image.
3. Choose **All Workers**, which is available when there is more than one worker, or choose a single worker.
4. Choose a time range of 5m, 15m, 30m, 1h, 1d, or 7d. Refresh for the latest data.

## Read the hardware charts

On a GPU endpoint, **Hardware** shows readings for **AVG GPU UTIL**, **AVG VRAM**, **AVG GPU MEMORY**, **AVG TEMP**, **MAX TEMP**, **TOTAL POWER**, **AVG CPU**, and **AVG MEMORY**. For one worker, the readings are **GPU UTIL**, **VRAM**, **GPU MEMORY**, **AVG TEMP**, **MAX TEMP**, **POWER**, **CPU**, and **MEMORY**. Charts below them follow GPU utilization, GPU memory used, GPU temperature, GPU power draw, CPU utilization, and CPU memory used. A CPU endpoint shows only the CPU and memory readings and charts.

With more than one worker on a GPU endpoint, choose **All Workers** and **Hardware**, then select **Compare Replicas**. **Real Time GPU Utilization by Worker** compares GPU utilization across the workers.

## Read the serving charts

For vLLM and SGLang endpoints, **Service** shows readings and charts for **Request Rate**, **TTFT p95**, **ITL p95**, **Token Throughput**, **TTFT SLO Compliance**, **Prefix Cache Hit Rate**, **Preemptions / sec**, and **Achieved TFLOP/s per GPU**. **TTFT p95** is the time to the first token, and **ITL p95** is the gap between tokens. Use them to tell whether callers wait too long for the first token, and add workers when they do. A vLLM-Omni endpoint shows its own set of serving charts.

## Set an alert

Open the **Settings** tab. The **Alerts** block manages the endpoint's alerts in Alert Management. An alert sends you an email when a monitored metric crosses its limit, so you can act before performance or availability suffers. See [Attach an alert to a resource](/tir/alerts/how-to/attach-an-alert-to-a-resource/).

## Check the charts

To follow what a worker reports, see [View logs and events](/tir/model-endpoints/how-to/view-logs-and-events/).