E2E CloudAlert Management FAQ
Setting up alerts
Section titled “Setting up alerts”Which resources can I set alerts on?
Section titled “Which resources can I set alerts on?”You can set alerts on an Instance, a VM, an Inference service, a Training Cluster, and a Load Balancer. Other resource types, such as an Auto Scaling Group, have no alert panel. See Alert metrics and conditions.
How many steps does it take to get alerts?
Section titled “How many steps does it take to get alerts?”Three: create a monitoring integration, create an alert that uses it, and attach the alert to a running resource. See Get started.
Does creating an alert start watching a resource?
Section titled “Does creating an alert start watching a resource?”No. Creating an alert only defines the rule. Attach it to a resource to start watching.
Why is Attach unavailable?
Section titled “Why is Attach unavailable?”Alerts attach only to a resource that is running. Start the resource, then attach. See Attach an alert to a resource.
Can I change the service of an alert?
Section titled “Can I change the service of an alert?”No. Service for alert is set when you create the alert, and everything else can change. For another service, create a new alert and delete the old one. See Manage alerts.
Why is the Alert tab missing on my Instance?
Section titled “Why is the Alert tab missing on my Instance?”The portal hides the Alert tab while an Instance is expired. On a running Instance it sits with the other tabs. See Attach an alert to a resource.
Why does an Inference service have no Alert tab?
Section titled “Why does an Inference service have no Alert tab?”Its alerts are in the Settings tab, under Alerts. See Attach an alert to a resource.
When an alert fires
Section titled “When an alert fires”How long must a condition last before an alert fires?
Section titled “How long must a condition last before an alert fires?”Five minutes before TIR sends the firing notification. The Status in the resource’s alert table reads Firing as soon as the condition is met. The alert resolves when the value is back within the threshold.
Can I set the condition for XID Error or Nodes Down?
Section titled “Can I set the condition for XID Error or Nodes Down?”No. XID Error always means Not Equal To 0, and Nodes Down always fires when more than 0 nodes are down. See Alert metrics and conditions.
Notifications and integrations
Section titled “Notifications and integrations”Where do notifications go?
Section titled “Where do notifications go?”To the monitoring integrations the alert uses: email to project members, a Slack channel, or a webhook URL. An alert can use several integrations. See What a notification contains.
Can I send an alert to an email address that is not a member of my project?
Section titled “Can I send an alert to an email address that is not a member of my project?”No. An email integration lists the members of the project, and each receives the email at the address of their account. To notify someone outside the project, give a webhook URL that forwards to them, or add them as a project member. See Create a monitoring integration.
How many headers can a webhook have?
Section titled “How many headers can a webhook have?”Up to 10 custom headers per integration. See Webhook requests.
Does TIR retry a webhook?
Section titled “Does TIR retry a webhook?”No. TIR sends each request once, waits up to 10 seconds for the connection, and ignores the answer. Answer with a 2xx status quickly.
Can I test an integration before I use it?
Section titled “Can I test an integration before I use it?”You can test a webhook with Test URL and Slack with Test URL or Test Token. Email has no test, so confirm it works when the first alert arrives. The webhook test leaves out the headers you added in the form. See Create a monitoring integration.
Why can I not delete a monitoring integration?
Section titled “Why can I not delete a monitoring integration?”You can delete an integration only after you remove it from every alert that uses it. See Manage monitoring integrations.
Deleting
Section titled “Deleting”What happens to attached resources when I delete an alert?
Section titled “What happens to attached resources when I delete an alert?”The alert is detached from all of them and stops sending notifications. To stop it on one resource only, detach it there. See Manage alerts.