--- title: Jobs Tab sidebar_label: Jobs --- import { List, Filter, Play, Clock, Terminal } from 'react-feather'; # Jobs Tab The **Jobs** tab is `squeue` in the browser. It shows every job Slurm currently knows about, with state counts and filters, so you can check a queue without SSH. }, { href: '#state-counters', label: 'State counters', icon: }, { href: '#filters', label: 'Filters', icon: }, { href: '#the-job-table', label: 'The job table', icon: }, { href: '#what-the-tab-cannot-do', label: 'What it cannot do', icon: }, { href: '#the-command-line-equivalents', label: 'CLI equivalents', icon: }, ]} /> --- ## Open the Jobs Tab 1. Go to **Training Cluster** in the TIR sidebar. 2. Open the cluster. 3. Select the **Jobs** tab. The data comes live from the Slurm controller each time the tab loads. --- ## State Counters Five cards across the top: | Card | What it counts | |------|----------------| | **RUNNING** | Jobs executing on nodes now | | **PENDING** | Jobs queued, waiting for resources | | **COMPLETED** | Jobs that finished successfully | | **FAILED** | Jobs that ended with an error | | **UNKNOWN** | Jobs in a state the tab does not break out separately | :::info Other Slurm states land under UNKNOWN Slurm has more states than these five. A job that is `CANCELLED`, `TIMEOUT`, `SUSPENDED`, `PREEMPTED`, `CONFIGURING` or `COMPLETING` shows as **Unknown** in the table and is counted under **UNKNOWN**. When you need the real state, use `squeue` or `sacct` — see [the command-line equivalents](#the-command-line-equivalents). ::: --- ## Filters Tabs below the counters narrow the table. Each carries its count: `ALL` · `RUNNING (n)` · `PENDING (n)` · `COMPLETED (n)` · `FAILED (n)` · `UNKNOWN (n)` Filtering happens in the page, so switching tabs is instant. A job in a state outside the five is only visible under **ALL**. --- ## The Job Table | Column | What it shows | |--------|---------------| | **Job ID** | Slurm's job ID. Use it with `scontrol show job` and `sacct` | | **Name** | The job name — from `--job-name` or the script filename | | **User** | Who submitted it. With [Login User Management](/docs/tir/SlurmCluster/connect/login-user-management) this is a real person; otherwise everything is `root` | | **Partition** | The [partition](/docs/tir/SlurmCluster/slurm-configuration/partitions) the job is in | | **Nodes** | Nodes allocated to the job | | **GPUs** | GPUs allocated | | **Run Time** | Elapsed time since the job started | | **Time Limit** | Maximum runtime — `UNLIMITED`, or `HH:MM:SS` / `D-HH:MM:SS` | | **State** | Running, Pending, Completed, Failed, or Unknown | | **Priority** | The scheduling priority, as a bar and a number | Empty queue reads **No jobs found.** :::note Read the priority number, not the bar Slurm's multifactor priorities are very large integers, while the bar is scaled to 100. In practice the bar sits at the top for nearly every job — the number beside it is the value that matters when comparing two pending jobs. ::: ### A job stuck in Pending Pending is normal — it means Slurm has not found resources matching the request yet. To find out why: ```bash squeue -j -o "%.18i %.9P %.30j %.8T %.10M %.20R" ``` The last column is the reason. Common ones: | Reason | Meaning | What to do | |--------|---------|------------| | `Resources` | Not enough free nodes or GPUs right now | Wait, or ask for less | | `Priority` | A higher-priority job is ahead of it | Wait, or use a partition with a higher priority tier | | `PartitionNodeLimit` | The job wants more nodes than the partition holds | Fix `--nodes`, or [add nodes to the partition](/docs/tir/SlurmCluster/slurm-configuration/partitions#edit-a-partition) | | `PartitionTimeLimit` | `--time` exceeds the partition's **Max time** | Lower `--time` or raise the partition's limit | | `ReqNodeNotAvail` | A requested node is down or drained | Check the [Nodes tab](/docs/tir/SlurmCluster/manage/nodes) | | `QOSMaxJobsPerUserLimit` | A QoS limit has been reached | Wait for your other jobs to finish | | `InvalidQOS` / `InvalidAccount` | The job names a QoS or account that does not exist | Correct the job script — TIR does not create QoS entries | Cross-check available capacity on the [Cluster Overview tab](/docs/tir/SlurmCluster/manage/overview-tab) — **Idle Nodes** and **Allocated GPUs** tell you at a glance whether the cluster is full or the request is unsatisfiable. --- ## What the Tab Cannot Do | | | |--|--| | **Cancel a job** | Not available here. Use `scancel ` over SSH | | **Hold or requeue** | Use `scontrol hold` / `scontrol requeue` | | **See job output** | The tab shows no `stdout`. Read your `--output` file on shared storage | | **Auto-refresh** | The tab does not poll. Use the refresh icon | | **Sort columns** | Rows come back in the controller's own order | | **See finished jobs from days ago** | Only what the controller still holds. Use `sacct` for history | :::tip Keep completed jobs visible longer By default Slurm drops completed jobs from `squeue` quickly, so they vanish from this tab too. Raise `MinJobAge` in [extra `slurm.conf`](/docs/tir/SlurmCluster/slurm-configuration/slurm-conf#useful-settings) — for example `MinJobAge=300` keeps them for five minutes. ::: --- ## The Command-Line Equivalents For anything the tab does not cover, SSH in: ```bash ssh root@ ``` ### Inspect the queue ```bash squeue # everything squeue -u $USER # your jobs squeue -p all -t PENDING # pending jobs in one partition squeue -o "%.18i %.30j %.8u %.8T %.10M %.6D %R" # custom columns incl. reason ``` ### Inspect one job ```bash scontrol show job # full detail: nodes, GRES, paths, exit code sstat -j # live resource usage of a running job ``` ### Job history and accounting ```bash sacct -j --format=JobID,JobName,State,ExitCode,Elapsed,MaxRSS,AllocTRES%40 sacct -u $USER -S today # your jobs since midnight sacct -S 2026-09-01 -E 2026-09-10 --format=JobID,JobName,State,Elapsed ``` `sacct` reads the accounting database, so it sees jobs long after they leave `squeue` — the right tool for "why did last night's run fail?". ### Control jobs ```bash scancel # cancel one scancel -u $USER # cancel all of yours scancel --state=PENDING -u $USER # cancel only your queued jobs scontrol hold # stop it being scheduled scontrol release # let it schedule again scontrol update job= TimeLimit=04:00:00 scontrol update job= Partition=debug ``` :::warning `scancel -u` cancels everything you own On a shared cluster with [per-user logins](/docs/tir/SlurmCluster/connect/login-user-management) that is scoped to you. If everyone logs in as `root`, `scancel -u root` cancels **the whole cluster's** jobs. Cancel by job ID, and give your team individual identities. ::: --- ## Related Resources - [Cluster Overview tab](/docs/tir/SlurmCluster/manage/overview-tab) - [Monitoring tab](/docs/tir/SlurmCluster/manage/monitoring) - [Nodes and GPU health](/docs/tir/SlurmCluster/manage/nodes) - [Manage partitions](/docs/tir/SlurmCluster/slurm-configuration/partitions) - [Troubleshoot jobs](/docs/tir/SlurmCluster/troubleshoot/jobs) - [Slurm `squeue` reference](https://slurm.schedmd.com/squeue.html)