Skip to main content

How to run a Jupyter Notebook inside Pytorch Container for Accelerated Machine Learning E2E GPU Wizard?

Launching an Instance based on Pytorch container with Jupyter-notebook Running​

Using myaccount portal​

  1. First, sign into the myaccount portal.

  2. Navigate to GPU Wizard:

    • Under the compute menu, click on GPU.
    • Then click on GPU Wizard.
    • For NGC Container Pytorch, click on Next under the actions column.
    • Choose the card according to the requirement; A100 is recommended.

    Image

  3. Choose the plan as per the requirement.

    • Optionally, you can add an SSH key (recommended) or subscribe to CDP backup.
    • Click on “Create my node”.
    • Wait for a few minutes and confirm that the node is in a running state.

    Image

  4. Open a terminal on your local PC and type the following command:


ssh -NL localhost:1234:localhost:8888 root@<your_node_ip>

Image

  1. The command usually will not show any output, which indicates that the command has run without any error.

  2. Go to a web browser on your local PC and hit the URL: http://localhost:1234

Image

  1. Congratulations! Now you can run your Python code inside this Jupyter notebook, which has Pytorch and all the libraries frequently used in machine learning preconfigured.

  2. To get the most out of GPU acceleration, use RAPIDS and DALI, which are already installed inside this container.

  3. RAPIDS and DALI accelerate tasks in machine learning, including data loading and preprocessing.

How to Run FashionMNIST Dataset Image Recognition Pytorch Notebook​

  1. Download the notebook by clicking on the following link on your local PC:
    FashionMNIST Notebook

  2. Upload the notebook in your Jupyter notebook environment.

  3. Choose Run All from the Cell Menu.

Image

  1. You can observe the whole workflow of machine learning and see an accuracy of approximately 63% after 5 epochs of training.

Image

Troubleshooting steps if notebook isn’t accessible​

  1. SSH into your node with the public IP:
    ssh username@<public-ip>

Image

  1. Check the output of the docker ps command.

Image

  1. If it shows a container is running using the Pytorch image, then you can use the following command in the terminal:
            cnid=$(docker ps |grep pytorch|head -1|awk '{print $1}');
            /usr/bin/nohup /usr/bin/docker exec $cnid
            /usr/local/bin/jupyter-notebook --no-browser --port=8888 --allow-root 
            --NotebookApp.token='' --NotebookApp.password='' 
            --NotebookApp.quit_button=False 
            --NotebookApp.notebook_dir=/home/pytorch &

Image

  1. If docker ps doesn’t list any container running, then we need to start the container first.

    i. Run docker ps -a

Image

  1. Now copy the container ID and run docker start <container_id>

Image

  1. Run the following command.
        cnid=$(docker ps |grep pytorch|head -1|awk '{print $1}');
        /usr/bin/nohup /usr/bin/docker exec $cnid /usr/local/bin/jupyter-notebook 
        --no-browser --port=8888 --allow-root --NotebookApp.token='' 
        --NotebookApp.password='' --NotebookApp.quit_button=False 
        --NotebookApp.notebook_dir=/home/pytorch &


Image

  1. Now create a ssh tunnel again in your local PC with ssh -NL localhost:1234:localhost:8888 root@<your_node_ip> and attempt to access the Jupyter notebook.

Image

Why doesn't my notebook have a password?​

The notebook is accessed via an SSH tunnel which creates a secure encrypted connection between the node and PC, making it accessible only for those who have SSH access. But if you want to set a password, you can do it the following way:

  1. Stop any existing Jupyter notebook processes inside the container.

  2. Edit the field –NotebookApp.password='Your password' in the above commands.

  3. To set it for the next boot, open the file /etc/rc.local in your server and set the password as shown below.

Image