Mount a PFS on a training cluster

Mount one or more PFS on every worker of a training cluster or a Slurm cluster. Each worker reads and writes at the paths you choose, ready for parallel access during the job.

Before you begin

You need a PFS in status OK, in the same project as the cluster; see Create a PFS. The cluster’s create screen lists every PFS in the project, and you can select the ones that read OK.

Choose the PFS when you create the cluster

On the create screen for a training cluster or a Slurm job, select one or more PFS to mount on every worker. Give each one its own mount path. A path must be unique across the Datasets, SFS, and PFS you choose for that cluster.

A training cluster job of the Slurm type needs at least one SFS or PFS selected; for other training cluster jobs, one is optional.

A TIR Slurm Cluster takes a PFS on its own create form, alongside SFS volumes and datasets, and mounts it on the login node as well as every worker. To add or remove one there, see Mount and unmount volumes.

Check what is mounted

After the cluster is created, its Volumes tab lists the PFS mounted on it, and you can filter the list to mounted or unmounted. The PFS’s own Services tab shows the same list under Training Cluster Deployments; see See what uses a PFS.

Design the job to read and write in parallel across its workers. Use larger blocks for sequential transfers, so the job uses a PFS’s bandwidth fully.

When a job needs more space, increase the size of a PFS, or mount the same PFS on an instance as well. For help with a path conflict or another message while you create the cluster, see Troubleshooting TIR Parallel File System.