Monitor an instance

Use an instance’s monitoring, logs, events, and alerts to catch a full disk before it stops your work, and to find out why an instance restarted.

Before you begin

• An instance in the Running state. Monitoring shows data only while the instance is running, and Logs and Instances Event stay empty while it is stopped or stopping. You can attach alerts only while it is running.
• An alert in the project, if you want to attach one. You can create it from the Alert tab.

Watch resource use on the Monitoring tab

Open the instance’s page and select Monitoring.

Two readings at the top tell you how full the instance’s storage is:

• Workspace Usage is the workspace disk at /home/jovyan. When it nears 100%, increase it on the Workspace Size tab while the instance keeps running. See Increase the workspace size.
• Ephemeral Storage is the rest of the container: every folder outside /home/jovyan and your mounts. Ephemeral means its contents are lost when the container restarts. The plan sets its size, and when the instance writes more than the plan allows, it restarts and everything in ephemeral storage is lost. Move large files into /home/jovyan or mounted storage before it fills. See Where your files live on an instance.

The charts follow CPU and RAM use over time, and on a GPU plan, GPU use, GPU memory, temperature, and power. Compare GPU Utilization with GPU Memory Used to tell a GPU that is busy computing from one that is only holding data in memory.

To look further back, choose 1h, 1d, 7d, or 30d.

Read and download the logs

Select Logs to see the output of the instance’s container, starting with the last 1,000 lines.

• To see a different amount, choose Last N Lines or Last N Seconds under Filter By, and set Value of N. None removes the filter.
• To keep the view current, set Auto Refresh to 10 Seconds, 30 Seconds, or 1 Minute.
• To keep a copy, click Download logs. All the logs are saved to your computer as a JSON file named after the instance.

See the instance’s events

Select Instances Event to see what has happened to the instance’s container recently, and why.

To find out why an instance restarted, look at Instances Event and Logs while it is running, and at the Ephemeral Storage and RAM Utilization figures on Monitoring.

Attach alerts

An alert watches one metric and warns you when it crosses the alert’s threshold. On an instance, an alert can watch CPU use, memory use, workspace disk use, ephemeral storage use, GPU temperature, or GPU power. An alert on ephemeral storage or memory use warns you before the instance runs out and is restarted.

1. Open the instance’s page and select Alert.
2. Choose the alerts under Select Alerts, and attach them. To create an alert first, use the link on the tab to open the alert form. Create an alert explains its thresholds and notifications.

You can detach an alert from the same tab. Alerts attached to an instance are removed when you delete the instance.

When a metric points to a problem, such as an instance that keeps restarting, see Troubleshooting TIR Instances.