E2E CloudTIR Object Storage FAQ
For step-by-step fixes, see Troubleshooting TIR Object Storage.
Choosing and naming
Section titled “Choosing and naming”What is the difference between Object Storage and Block Storage?
Section titled “What is the difference between Object Storage and Block Storage?”Object Storage mounts read-only, and many workloads can mount the same bucket at once. Use it for large, shared, or growing data that they read. A TIR Block Storage disk is read-write on one instance at a time. Use it when an instance needs to read and write its data. See Mount a bucket on an instance or Slurm Cluster and Mount a disk on an instance.
Can I convert a bucket to a Block Storage disk?
Section titled “Can I convert a bucket to a Block Storage disk?”No. A bucket stays a bucket. To keep the data on a disk, create a Block Storage disk and copy the data across. See Delete a bucket.
What is the difference between the alias and the bucket name?
Section titled “What is the difference between the alias and the bucket name?”The bucket name is the name the bucket has in EOS, and it is unique across all of EOS. The alias is the
TIR-side name that workloads use, such as the /datasets/<alias> folder on an instance. On the create
form the two start out the same. See Create a bucket.
Can I change the bucket name TIR suggests?
Section titled “Can I change the bucket name TIR suggests?”Yes. Bucket Name starts with a suggestion that begins with tir-bkt-, and you can type your own,
3 to 63 characters of lowercase letters, numbers, and hyphens. The alias copies it as you type. See
Create a bucket.
Can I rename a bucket or its alias?
Section titled “Can I rename a bucket or its alias?”No. Both names are fixed once the bucket is created, so choose names you want to keep. See Create a bucket.
Can I use my own existing bucket as a dataset?
Section titled “Can I use my own existing bucket as a dataset?”Not from Create Object Storage, which always creates a new bucket. An existing EOS bucket attached from elsewhere is listed as an Existing EOS Bucket. Encryption is not supported on it, and lifecycle rules work only on buckets TIR created. Deleting it from the list leaves the bucket itself in place. See Create a bucket.
What is the maximum size of a bucket?
Section titled “What is the maximum size of a bucket?”There is no fixed maximum. A bucket grows with your data, and billing follows the month’s peak usage. A single object can be as large as 5 TiB. See Upload data to a bucket.
Can another project use my bucket?
Section titled “Can another project use my bucket?”No. A bucket belongs to one project and one region, so another project creates its own bucket. See Create a bucket.
Who in my project can use a bucket?
Section titled “Who in my project can use a bucket?”The Owner, an Admin, a Project Manager, and a Project Lead can manage buckets. A Member needs a policy that allows datasets, and each permission in it opens up one kind of action:
- Read permission opens the list and the bucket’s tabs.
- Create permission lets the Member create buckets.
- Update permission lets the Member upload objects and change versioning, lifecycle rules, and locks.
- Delete permission lets the Member delete buckets, objects, and rules.
The portal shows only the actions your role allows. The same permission covers Block Storage, Shared File System, and Parallel File System. See Create a bucket.
Getting data in
Section titled “Getting data in”How do I upload a large dataset, for example 60 GB?
Section titled “How do I upload a large dataset, for example 60 GB?”Use the MinIO CLI or s3cmd. The portal uploads up to five files at a time, and it advises the MinIO CLI or s3cmd for files over 5 GB. One compressed archive uploads faster than thousands of small files. See Connect to a bucket with the MinIO CLI or s3cmd and Upload data to a bucket.
Where do I find the access keys and endpoint for my bucket?
Section titled “Where do I find the access keys and endpoint for my bucket?”On the bucket’s Overview tab, under Storage Details, and inside the commands on its Setup tab. See Find a bucket’s access keys and endpoint.
Which tools can I use with a bucket?
Section titled “Which tools can I use with a bucket?”Any S3-compatible tool: mc, s3cmd, s3fs, and the AWS and MinIO SDKs. The portal recommends the
MinIO CLI. See Connect to a bucket with the MinIO CLI or s3cmd.
How do I delete data in a bucket?
Section titled “How do I delete data in a bucket?”Delete objects on the Data Objects tab, or use mc rm or s3cmd del. To clear the whole bucket at
once, use Empty. See Browse, download, and delete objects
and Empty a bucket.
Mounting
Section titled “Mounting”Where does the bucket appear inside my instance?
Section titled “Where does the bucket appear inside my instance?”At /datasets/<alias>, where the alias is the bucket’s Alias (Dataset Name). A Slurm Cluster
mounts it at a path you choose. See
Mount a bucket on an instance or Slurm Cluster.
Why can I not write to the dataset folder inside my instance?
Section titled “Why can I not write to the dataset folder inside my instance?”A bucket always mounts read-only. Write through an S3 client with the bucket’s keys. You can also use a Block Storage disk, a Shared File System, or a Parallel File System mounted beside it. See Mount a bucket on an instance or Slurm Cluster.
Which TIR products can mount a bucket?
Section titled “Which TIR products can mount a bucket?”An instance, a Virtual Machine, and a Slurm Cluster can mount a bucket, and many workloads can mount the same bucket at once. See Mount a bucket on an instance or Slurm Cluster.
Can I mount a bucket on a running instance?
Section titled “Can I mount a bucket on a running instance?”Yes. Tick it on the instance’s Associated Datasets tab and select Update; a running instance restarts. See Mount a bucket on an instance or Slurm Cluster.
Does deleting an instance delete the buckets mounted on it?
Section titled “Does deleting an instance delete the buckets mounted on it?”No. A bucket stays. Object Storage is billed as its own product, so it keeps its storage charge until you delete it. See Delete a bucket.
Does a bucket use my instance’s disk?
Section titled “Does a bucket use my instance’s disk?”No. A bucket’s data is stored in Object Storage, separate from the instance’s own disk and its workspace. See Mount a bucket on an instance or Slurm Cluster.
Keys, access, and encryption
Section titled “Keys, access, and encryption”Who can see my bucket’s access keys?
Section titled “Who can see my bucket’s access keys?”Anyone in the project with the read permission for datasets can open the bucket and see its keys. Share them only with people who need them. See Find a bucket’s access keys and endpoint.
Can I regenerate or rotate a bucket’s keys?
Section titled “Can I regenerate or rotate a bucket’s keys?”No. The keys stay the same for as long as the bucket exists. See Find a bucket’s access keys and endpoint.
Can I share a bucket with other members of my project?
Section titled “Can I share a bucket with other members of my project?”Yes. A bucket belongs to its project. Anyone in the project whose role allows reading datasets can open it and see its keys. To share one object outside the project, use a presigned URL. See Find a bucket’s access keys and endpoint and Share an object with a presigned URL.
Is a bucket public?
Section titled “Is a bucket public?”No. A bucket is always private. You reach its data with the bucket’s keys, a presigned URL, or a mount from your own workload. To share one object outside the project, use a presigned URL rather than handing out the secret key. See Share an object with a presigned URL.
Is my data secured in transit?
Section titled “Is my data secured in transit?”Yes. Traffic to and from a bucket is secured with TLS, and the s3cmd settings on the Setup tab connect over HTTPS. See Connect to a bucket with the MinIO CLI or s3cmd.
Should I use E2E Managed or User Managed encryption?
Section titled “Should I use E2E Managed or User Managed encryption?”Use E2E Managed unless you must hold the keys yourself. With User Managed, you alone hold the key, so losing it loses the data. See Create an encrypted bucket.
Can I mount an encrypted bucket?
Section titled “Can I mount an encrypted bucket?”With E2E Managed encryption, yes: uploads, downloads, and mounts work as usual. With User Managed encryption, no. The Dataset Encryption dialog says the bucket cannot be mounted with other TIR services such as instances. See Create an encrypted bucket.
Can I add encryption to a bucket I already created?
Section titled “Can I add encryption to a bucket I already created?”No. You set encryption while you create the bucket, in Delhi only, and it stays as you set it. See Create an encrypted bucket.
Does encryption cost extra?
Section titled “Does encryption cost extra?”No. Encryption is free for both E2E Managed and User Managed buckets. See Create an encrypted bucket.
Versioning, locks, and lifecycle
Section titled “Versioning, locks, and lifecycle”What happens to old versions when I suspend versioning?
Section titled “What happens to old versions when I suspend versioning?”They stay available, and they keep being stored and billed. Suspending versioning only stops new versions from being created. See Use object versioning.
Can I add object lock to an existing bucket?
Section titled “Can I add object lock to an existing bucket?”No. You can turn object lock on only while you create the bucket, and turning it on turns versioning on permanently. See Set a default object lock on a bucket.
What is the difference between a governance lock and a legal hold?
Section titled “What is the difference between a governance lock and a legal hold?”A governance lock protects a version from deletes and overwrites, including those from lifecycle rules, until its retention ends. Only the Owner or an Admin can bypass it. A legal hold stays in place until someone removes it. It wins over governance: the version stays protected from deletion even after its retention has ended. See Lock one object version or place a legal hold.
Can I recover data a lifecycle rule deleted?
Section titled “Can I recover data a lifecycle rule deleted?”No. A lifecycle rule deletes objects permanently, so plan for it before you add the rule. See Add a lifecycle rule.
Can I have several lifecycle rules on one bucket?
Section titled “Can I have several lifecycle rules on one bucket?”Yes. Use different prefixes to keep different data for different lengths of time. Each prefix can have one rule. See Add a lifecycle rule.
Does replication copy the objects already in my bucket?
Section titled “Does replication copy the objects already in my bucket?”No. Replication copies only the new objects written to the bucket into a bucket in another region. See Replicate a bucket to another region.
Billing
Section titled “Billing”How is a bucket billed?
Section titled “How is a bucket billed?”A bucket is billed on the peak usage in each calendar month, in two slabs, with data transfer on top. You pay for the storage you use rather than a size set in advance. For rates, see E2E’s pricing page, which the Storage Pricing link on the create form opens too. For an estimate across services, use the E2E pricing calculator. See Create a bucket.
What does peak usage mean?
Section titled “What does peak usage mean?”It is the highest amount of storage the bucket reaches at any point in the month. The bill follows that peak rather than the average or the month-end amount, so data you delete late in the month still counts toward that month’s bill. Delete early, or use a lifecycle rule to clean up as you go. See Add a lifecycle rule.
Do old versions count toward my bill?
Section titled “Do old versions count toward my bill?”Yes. Every stored version counts, so a file overwritten ten times with versioning on leaves ten versions on the bill. Pair versioning with a lifecycle rule. See Use object versioning.
How do I keep the bill down?
Section titled “How do I keep the bill down?”Compress before uploading, and prefer efficient formats such as Parquet over CSV. Stage large intermediate work on a disk and move only the result into Object Storage. That keeps a short spike out of the month’s peak. See Upload data to a bucket.
Does a bucket cost anything after I delete it?
Section titled “Does a bucket cost anything after I delete it?”No. Deleting a bucket removes its charge. See Delete a bucket.