--- title: "Where your files live on an instance" description: "Which folders on a TIR Instance keep your files through a restart, a stop, or a delete, and where to put data, packages, and checkpoints." content_type: "concept" product: "tir-instance" platform: "TIR" url: "/tir/tir-instance/concepts/where-your-files-live/" --- # Where your files live on an instance A TIR Instance is a container started from an image, and where you save a file decides whether it is still there after a restart, a stop, or a delete. Only the workspace at `/home/jovyan` and the storage you mount keep their files when the container restarts; everything else goes back to the image's contents. Save your notebooks, data, checkpoints, and models to the workspace or to mounted storage. ## The four places a file can be ```mermaid flowchart LR I[Your instance] --> W["Workspace, /home/jovyan: kept through restarts, stops, and starts"] I --> E["Rest of the container: lost when the container restarts"] I --> L["/mnt/local, on some instances: deleted on stop unless it is a private cluster's volume"] I --> M["Datasets, SFS, and PFS: kept in their own storage"] ``` An instance holds files in four places: - The workspace, mounted at `/home/jovyan`. Files there are kept through restarts, stops, and starts, and are deleted with the instance: when you delete it, when a committed term ends with auto deletion, or when its project is deleted. - The rest of the container, called ephemeral storage: every folder outside `/home/jovyan` and the mounts. Its contents are lost each time the container restarts. - Local NVMe storage at `/mnt/local`, on plans that include it. A restart keeps it. A stop deletes everything on it, unless it is a private cluster's local volume. - Mounted datasets, Shared File Systems (SFS), and Parallel File Systems (PFS). Their data lives in that storage, outside the instance, so it is kept through restarts and when the instance is deleted. ## The workspace: `/home/jovyan` The workspace is a disk of its own, mounted at `/home/jovyan`, and the place for anything you want to keep. It is 30 GB by default, and you can set it from 30 GB to 15,000 GB when you create the instance. Later you can grow it on the **Workspace Size** tab while the instance runs. The limit is your account's limit in that region, which is 15,000 GB unless E2E has set another. The workspace can only grow. The first 30 GB are free, and each GB above that is billed while the instance exists, including while it is stopped. The rates are on [E2E's pricing page](https://www.e2enetworks.com/pricing). See [Increase the workspace size](/tir/tir-instance/how-to/increase-the-workspace-size/). When the workspace is full, writing under `/home/jovyan` fails with `no space left on device`. Increase the workspace size to fix it. You can do that while the instance keeps running. The workspace belongs to its instance. Deleting the instance deletes the workspace permanently. A clone of an instance starts with an empty workspace. ## Ephemeral storage: the rest of the container Every folder outside `/home/jovyan` and your mounts is the container's own disk. The instance's plan sets its size, and the **Monitoring** tab shows how full it is as **Ephemeral Storage**. On the C3.8GB and GDC.V100-8.120GB plans it is 50 GB; for another plan, read its size there. Its contents, including any package you install, are lost each time the container is replaced, which happens when: - you restart the instance, or stop and start it - you update its image or change its plan - you mount or unmount a dataset, SFS, or PFS while the instance is running - the instance runs out of ephemeral storage or memory Each of these also ends every process in the instance, such as a training run or a `tmux` session. The instance shows **Waiting** until it is ready again. Keep your work in `/home/jovyan` so it survives them. When the instance writes more to ephemeral storage than its plan allows, the container starts again and everything in ephemeral storage is lost. Files in `/home/jovyan` and mounted storage are kept. Write large downloads, datasets, and checkpoints to `/home/jovyan` or to mounted storage instead of `/tmp` or another folder outside them. An alert on ephemeral storage use warns you before the instance runs out; see [Monitor an instance](/tir/tir-instance/how-to/monitor-an-instance/). Files you write to `/dev/shm` are held in the instance's memory, so they count toward the plan's memory. ## Keep installed packages across restarts Packages you install yourself sit outside the workspace, so you install them again after a restart, a stop and start, an image update, or a plan change. The restart, mount, and unmount dialogs remind you of this. To keep packages, use one of these: - Install them from a start script, which runs each time the instance starts or restarts. See [Run start scripts on an instance](/tir/tir-instance/how-to/run-start-scripts-on-an-instance/). - Save the instance as an image, then launch or restore from that image. See [Save an instance as an image](/tir/tir-instance/how-to/save-an-instance-as-an-image/). - Launch the instance from a custom image that already has them. See [Create an instance from your own image](/tir/tir-instance/how-to/create-an-instance-from-your-own-image/). - Create a Python virtualenv or conda environment under `/home/jovyan`. Its files then survive restarts, stops, and starts. Lines that `conda init` or you add to `~/.bashrc` or `~/.profile` are lost each time the container is replaced, so activate the environment by its path, or from a start script. ## Local NVMe storage: `/mnt/local` `/mnt/local` is fast local NVMe storage. It comes with plans that include it, and with a private cluster's local volume. When your instance has it, **Plan Details** on the instance's **Overview** tab shows **Local NVMe Storage**. A restart keeps `/mnt/local`. When the storage comes with the plan, stopping the instance deletes everything on `/mnt/local`, and the stop dialog warns you about it. Deleting the instance deletes that local storage too, like the workspace. A private cluster's local volume is different: a stop keeps it, and deleting the instance detaches it so you can attach it to another instance when you create one. Copy anything you need from `/mnt/local` to the workspace or mounted storage before you stop the instance. ## Mounted storage: datasets, SFS, and PFS Mounted storage keeps its data in its own product, outside the instance: - Datasets appear at `/datasets/`. A dataset in an EOS bucket is read-only and can be mounted on many instances at once. A disk dataset is read-write and can be mounted on one instance at a time. - A Shared File System is read-write, and many instances can mount the same one at once. - A Parallel File System, in Delhi, is read-write, or read-only if you tick **Read Only** when you mount it. Several instances can mount the same one at once. You choose the mount path for an SFS or a PFS. A path inside the workspace, such as `/home/jovyan/shared`, puts the file system next to your workspace files. Deleting an instance unmounts its datasets and file systems and leaves their data in place, so you can mount them on another instance. Mounted storage is billed as its own product. To mount storage, see [Mount datasets and file systems on an instance](/tir/tir-instance/how-to/mount-datasets-and-file-systems/). ## What each action keeps | Action | Workspace (`/home/jovyan`) | Rest of the container | Mounted datasets, SFS, and PFS | | ------------------------------------------------------------------------------------ | ---------------------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | Restart | Kept | Lost | Kept | | Stop, then start | Kept | Lost | Kept | | Update the image, change the plan, or mount or unmount storage on a running instance | Kept | Lost | Kept | | Running out of ephemeral storage or memory | Kept | Lost | Kept | | Clone | The clone's workspace starts empty | New, from the image | The clone gets the same mounts; a disk dataset mounts on the clone only after you unmount it from the source, so unmount it first or leave it off the clone | | Delete | Deleted | Deleted | Unmounted and kept | For `/mnt/local`, see [Local NVMe storage](#local-nvme-storage-mntlocal): a restart keeps it, and a stop clears the local storage that comes with the plan. To move files in and out of the workspace, see [Copy files to and from an instance](/tir/tir-instance/how-to/copy-files-to-and-from-an-instance/). If files went missing after a restart, see [Troubleshooting TIR Instances](/tir/tir-instance/troubleshooting/tir-instance/).