Alert Management

Alert Management tells you when a resource in your project needs attention. You define a rule, such as “CPU Utilization is above 85 percent”, and TIR notifies you when a resource that uses the rule meets it. You can alert on an Instance, a VM, an Inference service, a Training Cluster, and a Load Balancer. Notifications go by email, to a Slack channel, or to an external endpoint as a webhook request.

Where to find it

In TIR, open your project. In the side menu, under PLATFORM SERVICES, select Alert Management. You can also reach a resource’s alerts from its own page. An Instance has an Alert tab, and a VM, a Training Cluster, and a Load Balancer each have an Alerts tab. On an Inference service, the alerts are in the Alerts section of the Settings tab.

How it works

Setting up alerts takes three steps:

1. Create a monitoring integration. It holds where notifications go: email addresses of project members, a Slack channel, or a webhook URL.
2. Create an alert. It holds the metric, the condition, and the severity, and uses one or more monitoring integrations.
3. Attach the alert to a running resource. The alert watches the resource from then on.

A monitoring integration can serve many alerts, and an alert can watch many resources of its kind. Both belong to one project, and an alert also belongs to one region: it watches only resources in that region.

What you can do

Set up notifications

• Create a monitoring integration for email or a webhook.
• Send alerts to Slack with a webhook URL or a bot token.
• Manage monitoring integrations: view, update, and delete.

Create and use alerts

• Create an alert.
• Attach an alert to a resource, and detach it.
• Manage alerts: view, update, and delete.

Understand the details

• Alert metrics and conditions: the metrics each service offers, severity, operators, and thresholds.
• What a notification contains.
• Webhook requests: the request and its JSON fields.

Core terms

Term	Meaning
Alert	A rule that sends a notification when a metric meets a condition.
Monitoring integration	A named set of destinations for notifications: email, Slack, or webhook.
Alert trigger type	The metric the alert checks, such as CPU Utilization or GPU Temperature.
Threshold value	The number the metric is compared with.
Operator	How the metric is compared with the threshold, such as Greater Than.
Severity	The priority of the alert: Critical, Warning, or Info.

Set up your first alert

Get started with Alert Management walks through the three steps once. For help with a missing notification, see Troubleshooting Alert Management. Short answers are in the FAQ.