How to run a Jupyter Notebook inside Pytorch Container for Accelerated Machine Learning E2E GPU Wizard?
Launching an Instance based on Pytorch container with Jupyter-notebook Running
Using myaccount portal
-
First, sign into the myaccount portal.
-
Navigate to GPU Wizard:
- Under the compute menu, click on GPU.
- Then click on GPU Wizard.
- For NGC Container Pytorch, click on Next under the actions column.
- Choose the card according to the requirement; A100 is recommended.
-
Choose the plan as per the requirement.
- Optionally, you can add an SSH key (recommended) or subscribe to CDP backup.
- Click on “Create my node”.
- Wait for a few minutes and confirm that the node is in a running state.
-
Open a terminal on your local PC and type the following command:
ssh -NL localhost:1234:localhost:8888 root@<your_node_ip>
-
The command usually will not show any output, which indicates that the command has run without any error.
-
Go to a web browser on your local PC and hit the URL:
http://localhost:1234
-
Congratulations! Now you can run your Python code inside this Jupyter notebook, which has Pytorch and all the libraries frequently used in machine learning preconfigured.
-
To get the most out of GPU acceleration, use RAPIDS and DALI, which are already installed inside this container.
-
RAPIDS and DALI accelerate tasks in machine learning, including data loading and preprocessing.
How to Run FashionMNIST Dataset Image Recognition Pytorch Notebook
-
Download the notebook by clicking on the following link on your local PC:
FashionMNIST Notebook -
Upload the notebook in your Jupyter notebook environment.
-
Choose Run All from the Cell Menu.
- You can observe the whole workflow of machine learning and see an accuracy of approximately 63% after 5 epochs of training.
Troubleshooting steps if notebook isn’t accessible
- SSH into your node with the public IP:
ssh username@<public-ip>
- Check the output of the
docker ps
command.
- If it shows a container is running using the Pytorch image, then you can use the following command in the terminal:
cnid=$(docker ps |grep pytorch|head -1|awk '{print $1}');
/usr/bin/nohup /usr/bin/docker exec $cnid
/usr/local/bin/jupyter-notebook --no-browser --port=8888 --allow-root
--NotebookApp.token='' --NotebookApp.password=''
--NotebookApp.quit_button=False
--NotebookApp.notebook_dir=/home/pytorch &
-
If
docker ps
doesn’t list any container running, then we need to start the container first.i. Run
docker ps -a
- Now copy the container ID and run
docker start <container_id>
- Run the following command.
cnid=$(docker ps |grep pytorch|head -1|awk '{print $1}');
/usr/bin/nohup /usr/bin/docker exec $cnid /usr/local/bin/jupyter-notebook
--no-browser --port=8888 --allow-root --NotebookApp.token=''
--NotebookApp.password='' --NotebookApp.quit_button=False
--NotebookApp.notebook_dir=/home/pytorch &
- Now create a ssh tunnel again in your local PC with
ssh -NL localhost:1234:localhost:8888 root@<your_node_ip>
and attempt to access the Jupyter notebook.
Why doesn't my notebook have a password?
The notebook is accessed via an SSH tunnel which creates a secure encrypted connection between the node and PC, making it accessible only for those who have SSH access. But if you want to set a password, you can do it the following way:
-
Stop any existing Jupyter notebook processes inside the container.
-
Edit the field
–NotebookApp.password='Your password'
in the above commands. -
To set it for the next boot, open the file
/etc/rc.local
in your server and set the password as shown below.