Mount datasets and file systems on a VM

Mount a dataset, a Shared File System (SFS), or a Parallel File System (PFS) on a TIR Virtual Machine, and its data appears in a folder inside the VM. Datasets mount under /datasets, and an SFS or a PFS mounts at a path you choose. A dataset or an SFS mount is added again at every boot, so the folder is there each time the VM starts.

Before you begin

• A VM in the project and region where your storage is, and permission to update VMs in the project. See TIR Access Control.
• A dataset that is ready or an SFS that is active, in that project and region, or a PFS whose Storage Status is OK. You can create each one from its tab on the VM’s page, or in advance: for an SFS, see Create an SFS.
• Your work saved. Mounting or unmounting a dataset, an SFS, or a PFS restarts a running VM.

Datasets, SFS, and PFS storage you mount are billed under their own products. They keep billing while the VM is stopped, and you delete them yourself to stop the charge. Deleting the VM detaches datasets and an SFS. They stay in your project, and you can mount them on another VM. For prices, see E2E’s pricing page.

Choose what to mount

A dataset’s Storage Type is EOS Bucket or Disk:

• A dataset in object storage (EOS Bucket) mounts read-only at /datasets/<dataset-name>. Choose it when the VM only reads the data.
• A disk dataset (Disk) mounts read-write at /datasets/<dataset-name>, on one VM at a time. Choose it when the VM needs to write to the data.
• A Shared File System is a shared, read-write file system that several VMs, instances, and the workers of a training job can mount at the same time. It mounts at the path you choose. Choose it when several machines need to read and write the same files together. A dataset is for data you mainly read.
• A Parallel File System mounts at the path you choose, and it is available in Delhi. To choose between a PFS and an SFS, see Parallel File System.

Mount storage when you create a VM

On the Create VM form, under Storage, the optional Other storage offers Shared File System, Datasets, and in Delhi Parallel File System. Choose the ones to mount, and for an SFS or a PFS also type a mount path that follows the mount path rules, such as /my-sfs or /my-pfs. To mount a PFS read-only, tick Read Only. For a dataset or SFS you still need to make, its Click here link opens the create page in a new tab. See Create a VM for the rest of the form.

Mount or unmount a dataset

1. In Manage VMs, select the VM’s name, then select the Associated Datasets tab.
2. Tick each dataset to mount, and clear each one to unmount.
3. Select Update. The change restarts the VM, so tick “Are you sure you want to continue updating datasets?” and select Update again.

The VM restarts, and each dataset’s State is Mounted or Unmounted when it is back.

A disk dataset mounts on one VM at a time. To use one that is mounted on another VM, unmount it there first.

Mount or unmount a Shared File System

1. In Manage VMs, select the VM’s name, then select the Shared File-System tab.
2. In the SFS’s row, type a Mount Path that follows the mount path rules, then select Mount.
3. Mounting restarts the VM. In Mounting SFS, tick “Are you sure you want to continue mounting SFS?” and select Continue.

The VM restarts, and the file system is Mounted when it is back. You mount one SFS at a time.

To unmount, select Unmount in the SFS’s row and confirm the same way. The VM restarts without it.

Mount or unmount a Parallel File System

Parallel File Systems are available in Delhi.

1. In Manage VMs, select the VM’s name, then select the Parallel File-System tab.
2. In the PFS’s row, type a Mount Path that follows the mount path rules. To mount it read-only, tick Read Only. Then select Mount. A PFS mounts once its Storage Status is OK.
3. Mounting restarts the VM. In Mounting PFS, tick “Are you sure you want to continue mounting PFS?” and select Continue.

The VM restarts, and the PFS is Mounted when it is back. To unmount, select Unmount in the same row, and confirm in the Unmounting PFS dialog the same way.

Any user on the VM can read or write a mounted PFS according to the owner, group, and other permissions of each file and folder. For sizing, deleting, and what is using a PFS, see Parallel File System.

Mount path rules

The mount path of an SFS or a PFS must:

• start with / and not end with /
• use only letters, digits, hyphens, and underscores, with no //
• differ from the path of any SFS or PFS already mounted on the VM

The paths /home/jovyan, /mnt/local, /datasets, and anything under /datasets/ are reserved, so choose another path. A path that matches a mounted SFS or PFS is refused with “file system path already mounted”. Mounting an SFS that is already mounted returns “SFS is already mounted”, and mounting a PFS that is already mounted fails with “pfs already mounted”.

Check the mounted folders

Once the VM is Running again, connect to it over SSH, as described in Connect to a VM over SSH, and list the folder:

ls /datasets/<dataset-name>

Replace <dataset-name> with the dataset’s name. For an SFS or a PFS, list the mount path you typed.

If a mount fails

A dataset must be ready and in the VM’s project and region. Otherwise, the mount is refused with “Some of the datasets are either invalid or not ready”. A mount that fails returns “Error while mounting/unmounting datasets on notebook”. For help with these messages, see Troubleshooting TIR Virtual Machines.

To create or manage a file system, see TIR Shared File System and Parallel File System.