TIR Object Storage

TIR Object Storage keeps your training data, checkpoints, and artifacts in a bucket in E2E Object Storage (EOS), an S3-compatible store. Many instances and clusters can read the same data from one place. A bucket grows by itself with the data you put in it. The portal labels it Object Storage (EOS Dataset), and it used to be called Datasets (EOS).

Choose Object Storage for data that many workloads read; choose TIR Block Storage for a disk that one instance reads and writes.

You can reach the data from the portal, from any S3-compatible tool, or by mounting the bucket on an instance, a Virtual Machine, or a Slurm Cluster. S3-compatible tools include the MinIO CLI (mc), s3cmd, s3fs, and the AWS and MinIO SDKs. A mounted bucket is always read-only.

Each bucket has a Bucket Name and an Alias (Dataset Name). The bucket name is the name it has in EOS, and it is unique across all of EOS. The alias is its name inside TIR. Workloads use the alias: a mounted bucket appears at /datasets/<alias> on an instance.

Where to find it

In TIR, open your project. In the side menu, under Storage, select Object Storage (EOS Dataset). You see the buckets of the current project and region.

What you can do

Create and connect

• Create a bucket with a name, an alias, and optional encryption, versioning, and object lock.
• Create an encrypted bucket and choose who holds the key.
• Find a bucket’s access keys and endpoint to connect an S3 tool or SDK.
• Connect to a bucket with the MinIO CLI or s3cmd using the ready-made commands on the Setup tab.

Work with data

• Upload data to a bucket from the portal or an S3 tool.
• Browse, download, and delete objects.
• Mount a bucket on an instance or Slurm Cluster to read it as a folder.
• Share an object with a presigned URL while the secret key stays with you.

Protect and manage data

• Use object versioning to keep earlier versions of an object.
• Set a default object lock on a bucket.
• Lock one object version or place a legal hold.
• Add a lifecycle rule to delete objects after a number of days.
• Replicate a bucket to another region.

Empty and delete

• Empty a bucket to remove every object and keep the bucket.
• Delete a bucket when you are done with it.

Use the API

The TIR API reference lists the dataset operations in Dataset.

Where to start

Start with Get started with TIR Object Storage, which creates a bucket and uploads a first file. For help with a message, see Troubleshooting TIR Object Storage. The TIR Object Storage FAQ has short answers about billing, access, and limits.