Skip to main content

Volume Mounts & Data Persistence in TIR Nodes

Introduction​

This document outlines how volume mounts work and the integration of Datasets (both EOS & Disk) with TIR nodes.

Data Persistence in TIR Nodes​

By default, TIR nodes provide a home directory (/home/jovyan) where files and data can be stored. Data written to this directory is persistent, meaning you can stop your node and start again at any time, and still have access to your files.

Note

Any files stored outside the home directory do not persist unless they are saved to a mounted volume (see the next section for more details).
Hence, the packages that you manually install in the node will need to be re-installed after the node is restarted.

Volume Mounts in TIR​

Volume mounts are critical for extending the storage capacity of your node and for integrating datasets from external sources. In TIR, volume mounts are typically used for the following reasons:

  • Access to large datasets: Datasets can be stored in external volumes and mounted to the node, enabling access without manually downloading files every time.
  • Persistent storage: When working with sensitive or large data, mounting a Disk or another external storage ensures the data remains even if you stop your nodes.
  • Scalability: For multi-node operations, mounting volumes ensures data is shared efficiently between nodes.

Datasets​

In TIR, datasets are containers for sharing, storing, and accessing data across different environments. Depending on the storage backend, datasets can be mounted on nodes with different access modes and purposes. Mounted Volumes are visible under the Associated Datasets section of the node.

EOS (E2E Object Store)​

EOS (E2E Object Store) is ideal for managing large datasets. EOS can be mounted on multiple nodes, allowing read-only access to the data from within the node environment.

How to Mount​

During Node Creation​

  1. In the "Datasets" section, select the desired EOS dataset(s) you wish to attach to your new node.
  2. Complete the node creation process. The selected EOS dataset(s) will be mounted to the node.

After Node Creation​

  1. Navigate to the "Associated Datasets" section of the Node.
  2. Select the desired EOS dataset(s).
  3. Click on Mount. The EOS dataset(s) will be mounted and accessible from within the node.

Disk​

The High Performance Disk storage backend is designed for fast high-speed data access and high throughput. Disk-based datasets allow for only single read-write access, and hence can only be attached to a single node at a time.

How to Mount​

During Node Creation​

  1. In the "Datasets" section, select the disk-based dataset you wish to attach to your new node.
  2. Complete the node creation process. The selected disk-based dataset will be mounted to the node.

After Node Creation​

  1. Go to the "Associated Datasets" section of the Node.
  2. Select the desired disk-based dataset(s).
  3. Click on Mount. The disk-based dataset will be mounted and accessible from within the node.

Access Modes​

  • EOS: Supports read-only access, allowing multiple nodes to mount and read the same dataset simultaneously.
  • Disk: Supports single read-write access, meaning a disk can only be mounted on one node at a time.
Note

Both EOS and disk-based datasets are mounted at the path /datasets/dataset-name.