E2E CloudTIR Parallel File System FAQ
Choosing a PFS
Section titled “Choosing a PFS”What is the difference between a PFS and an SFS?
Section titled “What is the difference between a PFS and an SFS?”Both are shared storage that you mount on instances and training jobs in one project. An SFS is sized in GB, from 250 GB, and is offered in Delhi and Chennai. A PFS is sized in whole TB, from 1 TB up to 250 TB by default, and is offered in Delhi. It shows an expected bandwidth of 1 GB/s for every TB. See Shared File System and Parallel File System.
| File system | What it is for |
|---|---|
| SFS | Everyday shared access from many workers at once, such as a distributed app or a shared workspace. |
| PFS | Workloads where many workers read and write at once and need more bandwidth than SFS, such as training across many instances. |
How do I know if I need a PFS instead of an SFS?
Section titled “How do I know if I need a PFS instead of an SFS?”Consider a PFS when your workload needs the highest throughput, runs parallel computing such as training across many instances, or needs steady performance while many workers read and write at once. Compare the price of each on E2E’s pricing page. See What is the difference between a PFS and an SFS?.
Can I convert an existing SFS to a PFS?
Section titled “Can I convert an existing SFS to a PFS?”No. To move the data, create a new PFS and copy the data across.
My instance’s own storage is not enough. What are my options?
Section titled “My instance’s own storage is not enough. What are my options?”Mount a Shared File System or a Parallel File System on the instance for more space. See Mount datasets and file systems on an instance and Create a PFS.
Size and performance
Section titled “Size and performance”What is the maximum size of a PFS?
Section titled “What is the maximum size of a PFS?”You can make a PFS as large as your account’s limit, 250 TB by default. To go above it, ask for a higher limit from the create or increase screen. The request opens a support ticket with E2E, and your limit stays as it is until E2E changes it. See Create a PFS.
Is there a minimum PFS size?
Section titled “Is there a minimum PFS size?”Yes, 1 TB. You set the size in whole TB on both the create and increase screens. See Create a PFS.
Do I need to unmount a PFS before increasing its size?
Section titled “Do I need to unmount a PFS before increasing its size?”No. You can increase the size while an instance, a training cluster deployment, or a Slurm cluster has the PFS mounted. An instance using it sees the new size while it keeps running. See Increase the size of a PFS.
What bandwidth can I expect from a PFS?
Section titled “What bandwidth can I expect from a PFS?”It scales with the size you choose. The create screen shows 1 GB/s of expected bandwidth for every TB you choose, for example 5 GB/s for a 5 TB PFS. For current prices, see E2E’s pricing page.
Why is my PFS not performing as expected?
Section titled “Why is my PFS not performing as expected?”The common causes, and what to check for each, are in I/O is slower than expected.
Mounting and access
Section titled “Mounting and access”Can more than one instance mount the same PFS at once?
Section titled “Can more than one instance mount the same PFS at once?”Yes. Each instance can mount it read-write or read-only. See See what uses a PFS.
What happens to my data when I unmount a PFS?
Section titled “What happens to my data when I unmount a PFS?”Unmounting only removes the PFS from that instance. The PFS and its files stay as they are until you delete the PFS. See Mount a PFS on an instance.
Can I mount a PFS to an instance in a different region?
Section titled “Can I mount a PFS to an instance in a different region?”No. A PFS mounts on instances in its own region, and PFS is available in Delhi. See Mount a PFS on an instance.
Can another project use my PFS?
Section titled “Can another project use my PFS?”No. A PFS belongs to one project. Parallel File System lists that project’s PFS, and the project’s own instances, training clusters, and Slurm clusters can mount it. Create a PFS in each project that needs one. See Create a PFS.
Do VM users need a UID and GID in the 1000 to 1050 range to access a PFS?
Section titled “Do VM users need a UID and GID in the 1000 to 1050 range to access a PFS?”No. That range applies inside a container. On a VM, any user with the right file permissions can access a mounted PFS. See Access and permissions.
How do I give several container users the same access to a PFS?
Section titled “How do I give several container users the same access to a PFS?”Put them in one group with a GID between 1000 and 1050, and set the shared PFS directory’s group to that group. See Access and permissions.
Should I limit which instances have a PFS mounted?
Section titled “Should I limit which instances have a PFS mounted?”Yes. Keep a PFS mounted only on the instances, training clusters, and Slurm clusters that need it. Check its Services tab from time to time to see what still has it mounted. See See what uses a PFS.
Billing and backups
Section titled “Billing and backups”Am I billed for a PFS whose creation ends in error?
Section titled “Am I billed for a PFS whose creation ends in error?”No. Billing starts only once a PFS reaches OK, so a PFS that never reaches OK is free. Delete it from its Actions menu. See Troubleshooting TIR Parallel File System.
How is a PFS billed?
Section titled “How is a PFS billed?”TIR bills a PFS by the hour for the size you set, mounted or unmounted. Billing starts once the PFS reaches OK and stops when you delete it. After an increase, billing moves to the new size once the increase completes. For current prices, see E2E’s pricing page.
How do I back up PFS data?
Section titled “How do I back up PFS data?”Copy anything that matters to Datasets, another PFS, or another SFS before you delete a PFS, since deleting it permanently removes all of its data. For large amounts of data, use rsync or a parallel copy tool. See Delete a PFS.