E2E CloudParallel File System
Parallel File System (PFS) is fast shared storage that you create once and mount on instances, training clusters, and Slurm clusters. Many workers can then read and write it at the same time.
TIR also offers Shared File System (SFS), shared storage you mount the same way. SFS suits shared access from many instances at once, such as a shared workspace. PFS suits parallel reads and writes at high throughput, such as training across many instances. For how the two compare in size, region, and bandwidth, see What is the difference between a PFS and an SFS?.
Where to find it
Section titled “Where to find it”In TIR, open your project. In the side menu, under Storage, select Parallel File System, which sits after Shared File System.
What you can do
Section titled “What you can do”Create and connect
Section titled “Create and connect”- Create a PFS with a name and a size in TB.
- Mount a PFS on an instance, at creation or afterward.
- Mount a PFS on a training cluster or a Slurm cluster.
Lifecycle
Section titled “Lifecycle”- Increase the size of a PFS when you need more space.
- Delete a PFS when you have finished with it.
Monitor
Section titled “Monitor”- See what uses a PFS: the instances, training clusters, and Slurm clusters that have it mounted.
Where to start
Section titled “Where to start”Start with Get started with TIR Parallel File System, which creates a PFS and mounts it on an instance. Short answers are in the TIR Parallel File System FAQ, and help with each message is in Troubleshooting TIR Parallel File System.