Skip to main content

How to Launch GPU H100 Notebook

Select from pre-built or one of your own images. Next, you choose GPU H100 plan and click on the create button.

Notebook options​

  • Disk Size: Each TIR Notebook can have a disk size of up to 5000GB. The default is 30GB. The selected disk will be mounted at /home/jovyan in your notebook environment. We recommend using this path as your workspace, so in case of restarts, your content will be persistent. Since TIR is container-native, the changes that you make to any other paths on the notebook will not be persisted on restarts. You can extend the disk size after the start of the notebook as well. This workspace will be deleted when the associated notebook is deleted.
Note

Please raise a support ticket if you need more than 1TB of disk workspace.

  • Local NVME Storage: Only available for GPU H100 plans. This fast local storage will be available at /mnt/local and only for the duration of the run. We recommend using this path when you need faster writes (e.g., save model checkpoints) or reads. Be sure to move this data to the EOS bucket or under /home/jovyan before shutting down the notebook. This type of storage is fixed and cannot be expanded at any time during the notebook cycle.
Note

By default, you receive 1000GB. The size of the local storage is fixed and cannot be altered.

  • Plan (Pricing): You can choose between an hourly or committed plan. We recommend using committed plans as they offer discounts and also may offer access to local NVME storage (for H100 plans only).

  • Notebook Image: TIR environments are container-native. You can use pre-built images with well-known frameworks like PyTorch, Transformers, or customize the pre-built images. You can make your own images TIR-compatible using the image builder utility. We recommend starting with pre-built images. In case you need to install packages from pip or apt-get, we recommend doing so from a Jupyter Notebook (.ipynb) or maintaining requirements.txt.

  • Delete Notebook: When a notebook is deleted, all the resources associated with it will be deleted, including the workspace (disk).

Steps to Create GPU H100 Notebook​

To create a Notebook, you have to click on Create Notebook, which is at the left corner of the page.

Create Notebook

After clicking on the Create Notebook button, a page will appear. Now enter the Notebook name, choose the Notebook Image, and select the GPU H100 machine to run your service, and then choose the plan. After that, click on the Create button.

Notebook Creation

GPU H100 Creation

After entering all the details (Disk Size, Datasets, and SSH key), you will click on the create button for creating a notebook.

Enable SSH

Enable SSH Step 1

After clicking on the create button, your GPU H100 will be created and it will be shown like this.

GPU H100 Created

Notebook Details​

Overview​

You can see the Notebook Details and Plan Details under the Overview tab.

Overview Details

Disk Size​

You can see the details of disk size and also change the Disk size as per your requirements.

Disk Size Details

For updating the disk size, you have to change the disk size and then click on the update button.

Update Disk Size

Metrics​

You can see the Metrics graph in CPU Utilization, Memory Utilization & Interval.

Metrics Graph

You can see the one-month activity as per your requirement in days & hours.

Monthly Activity

Associated Datasets​

You can also see the Associated Datasets with two different datasets - Mounted & Unmounted. You can also Unmount.

Associated Datasets

SSH Key​

You can see the SSH Key Details under the SSH key tab.

SSH Key Configuration

Note

Only one SSH key can be added to a notebook.

Steps to Verify the Configuration of the GPU H100 Notebook Using the Terminal​

  1. Create your notebook with SSH key.
  2. You can access your GPU H100 notebook by clicking the SSH Access icon.

Access SSH

  1. After clicking this SSH access icon, you will see the command details section.

Access SSH Command Details

  1. To access your GPU H100 notebook, use this command: ssh root@ip.

SSH Command

  1. To check the GPU resources, use this command: nvidia-smi.

NVIDIA SMI Command

NVIDIA SMI Output

Verify the Configuration of the GPU H100 Notebook Using the Lab URL​

  1. To launch the GPU H100 notebook, click on Jupyter.

Launch Jupyter

  1. After clicking on Jupyter, you can see that page and click on Python3.

Launch Python 3

Launch Python 3 Step 2

  1. To check the GPU resources, use this command: !nvidia-smi and press Shift + Enter.

Check GPU Resources

NVIDIA SMI Check

Notebook Actions​

You can see the actions like Launch Notebook, Stop, Update Notebook, and Delete.

Notebook Actions

Launch Notebook​

After clicking on Launch Notebook, the Notebook will be launched and it should be visible like this.

Notebook Launched

Notebook Launched Step 2

Stop​

Note

You cannot stop the GPU H100 notebook.

Cannot Stop

Delete Notebook​

To delete the Notebook, you have to click on the Delete button.

Delete GPU Notebook