Notebooks

TIR Notebooks are fully collaborative environments that make AI development possible. They combine power of containers, jupyter labs and AI/ML frameworks to create a readily usable workspace for you and your entire team.

Some of the most common use cases are:

Run a script or notebook to fine-tune a Large Language Model (LLM) on single GPU using pytorch or huggingface train
Run a script or notebook to tokenize and fine-tune LLMs or Diffusion models with with multiple GPUs (single machine) using deepspeed and accelerate
Open and run a jupyter notebook (.ipynb) from the platforms like github, kaggle or collab
Download and review datasets stored on TIR or other platforms like huggingface
Download and test models like stable diffusion, or any LLM

Note

A TIR notebook is fully functional coding environment.

If you prefer to work with command line (shell) over jupyter labs, you can configure ssh on a notebook. This way you can upload your data using sftp or sync your code with git tools and run the scripts as you would on your local system.

Getting Started

Go to TIR Dashboard
Create or Select a project
Click on Notebooks in side-bar section
Click CREATE NOTEBOOK
Choose an appropriate name for your notebook
Select a notebook type as NEW NOTEBOOK. If you want to open a notebook found on other platforms like github or google collab, you can choose IMPORT NOTEBOOK option which allows entering the URL of the target notebook.
Next, you can select from pre-built or one of your own images. For the sake of simplicity, Click on Pytorch 2 option
Next, you can choose a CPU or GPU plan. Feel free to choose Free Tier plan for this exercise
(Optional) Set Enable SSH Access switch to enabled and add or select your SSH key.
Leave the rest of the options as-is and click CREATE
Next, you will see a new notebook appear in the list of notebooks. Wait for the notebook to come to a ready state.
When the notebook is ready, you will see both Jupyter labs and SSH options (if configured). Choose any of these to access the notebook environment and work on your magic.

Notebook options

TIR notebooks are extremely powerful and flexible. While most configurations have a default to make our life eaiser, sometimes you may need to tweak the knobs. The following are the configurations that you can tweak in a notebook environments:

Enable SSH: You can enable SSH access on the notebook using public key or password (not recommended). If you decide to enable ssh after starting a notebook, you will have to first stop the notebook before you make changes.
Disk Size: Each TIR Notebook can have a disk size upto 1000GB. The default is 30GB. The selected disk will be mounted at /home/jovyan in your notebook environment. We recommend using this path as your workspace so in case of restarts, your content will be persistent. Since, TIR is container-native, the changes that you make to any other paths on the notebook will not be persisted on restarts. You can extend the disk size after the start of notebook as well. This workspace will be deleted when the associated notebook is deleted.

Note

Please raise a support ticket if you need more than 1TB of disk workspace.
Local NVME Storage: Only available for H100 plans. This fast local storage will be available at /mnt/local andonly for the duration of run. We recommend using this path when you need faster writes (e.g. save model checkpoints) or reads. Be sure to move this data to EOS bucket or under /home/jovyan before shutting down the notebook. This type of storage is fixed and can not be expanded at anytime during the notebook cycle.
Plan (Pricing): You can choose between an hourly or committed plan. We recommend using committed plans as they offer discounts and also may offer access to local NVME storage (for H100 plans only).
Notebook Image: TIR environments are container-native. You can use pre-built images with well known frameworks like pytorch, tranformers or customise the pre-built images. You can make your own images TIR-compatible using image builder utility. We recommend starting with pre-built images. In case you need to install packages from pip or apt-get, we recommend doing so from a jupyter notebook (.ipynb) or maintaining requirements.txt.
Configuration: TIR offers a variety of cpu and gpu options. We recommend using A100 or H100 for best performance.
Update Notebook: You can upgrade or downgrade both the configuration (e.g. upgrade from cpu to gpu) and Plan (e.g. hourly to commited) of a notebook if desired. This is useful option when restarting notebooks and the original hardware plan (gpu) on the notebook is no longer available.
Stop Notebook: If the plan and configuration allows, you can stop a notebook and restart. In case of hourly plan, you will not be billed for the GPU or GPU when notebook is in a stopped state. However, if your disk usage is beyond free tier, you will be charged for it.
Delete Notebook: When a notebook is deleted, all the resources associated with it will be deleted including the workspace (disk).

Notebook Statuses

Waiting: The notebook instance is being deployed on the hardware of your choise.
Running: The notebook is active and you can use either jupyter labs or ssh (if enabled) to access it.
Stopped: The notebook is not assigned to any machine. However, the workspace (disk mounted at /home/jovyan) will continue to exist until you delete the notebook. Depending on the size of the disk, you will charged for the usage.

How to create Notebook ??

To create a Notebook you have to click on Create Notebook which is at the left corner of the page.

After clicking on the Create Notebook button a page will appear, now enter Notebook name, choose the Notebook Image and select the machine on which you want to run your service, and then choose the plan, After that click on Create button.

After entering all the details Disk Size, Datasets and ssh key you will click on the create button for creating a notebook.

After clicking on the create button your GPU will be created and it will be shown like this.

Notebook Details

Overview

You can see the Notebook Details, Plan Details under Overview tab.

Disk Size

You can see the details disk size and also You can change the Disk size as per your requirements.

For updating the disk size you have to change the disk size and then click on update button.

Metrics

You can see the Metrics graph in CPU Utilization , Memory Utilization & Interval.

You can see the one month activity as per your requirement in days & hours.

Associated Datasets

You can also see the Associated Datasets with two different datasets- Mounted & Unmounted. You can also Unmount.

SSH Key

You can see the SSH Key Details under ssh key tab.

Update SSH Key

Note

Only one SSH key can be added to a nLaunch Notebook from Sidebar

You can launch the notebook from the left side of the Notebook name .

otebook

Note

When user want to change ssh key kindly first stop the notebook then change the ssh key

Add SSH Key After Notebook Creation

Note

When user want to add ssh key after Notebook creation, kindly first stop notebook then add ssh key

Notebook Actions

You can see the actions like Launch Notebook, Stop, Update Notebook, Delete.

Launch Notebook

After clicking on Launch Notebook, Notebook will be launched and it should be visible like this.

Stop

For Stopping the Notebbok you have to click on Stop button and the Notebook will be stopped.

Update Notebook

You can update Notebook, For updating the Notebook You have to click on Update button.

Note

Notebook must be in Stop state before updating the Notebook.

Delete Notebook

For Deleting the Notebook you have to click on Delete button.