How to Launch GPU H100 notebook

Select from pre-built or one of your own images, Next you choose GPU H100 plan and click on create button.

Notebook options

  • Disk Size: Each TIR Notebook can have a disk size upto 5000GB. The default is 30GB. The selected disk will be mounted at /home/jovyan in your notebook environment. We recommend using this path as your workspace so in case of restarts, your content will be persistent. Since, TIR is container-native, the changes that you make to any other paths on the notebook will not be persisted on restarts. You can extend the disk size after the start of notebook as well. This workspace will be deleted when the associated notebook is deleted.

    Note

    Please raise a support ticket if you need more than 1TB of disk workspace.

  • Local NVME Storage: Only available for GPU H100 plans. This fast local storage will be available at /mnt/local andonly for the duration of run. We recommend using this path when you need faster writes (e.g. save model checkpoints) or reads. Be sure to move this data to EOS bucket or under /home/jovyan before shutting down the notebook. This type of storage is fixed and can not be expanded at anytime during the notebook cycle.

    Note

    By default, you receive 1000 (GB). The size of the local storage is fixed and cannot be altered.

  • Plan (Pricing): You can choose between an hourly or committed plan. We recommend using committed plans as they offer discounts and also may offer access to local NVME storage (for H100 plans only).

  • Notebook Image: TIR environments are container-native. You can use pre-built images with well known frameworks like pytorch, tranformers or customise the pre-built images. You can make your own images TIR-compatible using image builder utility. We recommend starting with pre-built images. In case you need to install packages from pip or apt-get, we recommend doing so from a jupyter notebook (.ipynb) or maintaining requirements.txt.

  • Delete Notebook: When a notebook is deleted, all the resources associated with it will be deleted including the workspace (disk).

Steps to create GPU H100 notebook

To create a Notebook you have to click on Create Notebook which is at the left corner of the page.

../_images/Notebook1.png

After clicking on the Create Notebook button a page will appear, now enter Notebook name, choose the Notebook Image and select the GPU H100 machine to run your service, and then choose the plan, After that click on Create button.

../_images/Notebook2.png ../_images/createionGPUH100.png

After entering all the details Disk Size, Datasets and ssh key you will click on the create button for creating a notebook.

../_images/Enablessh.png ../_images/EnableSSh1.png

After clicking on the create button your GPU H100 will be created and it will be shown like this.

../_images/GPU_H100.png

Notebook Details

Overview

You can see the Notebook Details, Plan Details under Overview tab.

../_images/overview1.png

Disk Size

You can see the details disk size and also You can change the Disk size as per your requirements.

../_images/disk.png

For updating the disk size you have to change the disk size and then click on update button.

../_images/UpdateDisksize.png

Metrics

You can see the Metrics graph in CPU Utilization , Memory Utilization & Interval.

../_images/GPU_Notebooks_H100_Metrics.png

You can see the one month activity as per your requirement in days & hours.

../_images/monthlyActivity.png

Associated Datasets

You can also see the Associated Datasets with two different datasets- Mounted & Unmounted. You can also Unmount.

../_images/AssociatedDataset.png

SSH Key

You can see the SSH Key Details under ssh key tab.

../_images/ConfigureSShkey.png

Note

Only one SSH key can be added to a notebook

Here are the Steps to verify the configuration of the GPU H100 notebook using the terminal

  1. Create your notebook with ssh key.

  2. You can access your GPU H100 notebook by clicking SSH Access icon.

../_images/AccessSSH.png

  1. After clicking this ssh access icon, you can able to see the command details section.

../_images/AccessSSH2.png

  1. To access your GPU H100 notebook, use this command. ssh root@ip.

../_images/connectssh1.png

  1. To check the GPU resources, use this command nvidia-smi.

../_images/nvidia.png ../_images/nvidia2.png

Or you can verify the configuration of the GPU H100 notebook using the Lab URL

  1. To launch GPU H100 notebook click on jupter.

../_images/launch.png

  1. After clicking on jupter you can see that page and click on Python3.

../_images/launch1.png ../_images/launch2.png

  1. To check the GPU resources, use this command ! nvidia-smi and press shift + enter.

../_images/launch3.png ../_images/launch4.png

Notebook Actions

You can see the actions like Launch Notebook, Stop, Update Notebook, Delete.

../_images/Actions.png

Launch Notebook

After clicking on Launch Notebook, Notebook will be launched and it should be visible like this.

../_images/Launchnotebook.png ../_images/Notebook13.png

Stop

Note

You can not stop GPU H100 notebook.

../_images/Cannotstop.png

Delete Notebook

For Deleting the Notebook you have to click on Delete button.

../_images/DeleteGPU.png