Create notebooks instance with your own container image

TIR platform is container-native, designed to give you performance and flexibility advantage of the container eco-system in AI/ML without the infrastructure overhead.

You can create notebook instances based on your own custom container. Using a custom container lets you change the notebook environment for your specific needs. You can extend the TIR provided containers and add a set of packages or libraries that you need for your training pipelines.

If you are thinking - why I can’t I install packages in a running notebook as I already have root access? - the answer is you can but the TIR notebooks run inside containers and only the workspace and data volumes (e.g. /data, /home/jovyan) are persisted when you restart a notebook.

Steps to use custom container based notebooks:

Build a container image
Push the container image to the registry
Create notebook with custom container

1. Build a container image

TIR provides a number of container environments for most commonly used frameworks. We recommend patching the TIR provided images - as those are already configured to be compatible with TIR notebooks - but you may choose to go your own way given the following requirements are met.

The docker image must be built for the linux/amd64 platform. To ensure this, do one of the following:
- Add a FROM instruction to your Dockerfile:
```
FROM --platform=linux/amd64 <image>
```
- Specify the platform as a CLI flag when building the image:
```
$ docker build --platform=linux/amd64
```
Your image is pushed to a repository that is accessible by TIR
The container exposes 8888 port and runs a jupyter labs service on that port.

If these requirements are not met, you can proceed to create your own container image by patching the TIR provided images.

TIR’s pre-built environment images can be customised with any of the three options:

Patching using the Dockerfile
Patching an instance of the container and converting it to an image
Using TIR’s Image Builder Utility

1.1 Patching using the Dockerfile

First way to to build your container image is by writing your own custom Dockerfile and using any of the TIR provided images as the base image.

This way, you can create your custom container image that extends the functionality of TIR provided images, thereby making your container image compatible with TIR Notebooks.

Use the example guides below to write your custom Dockerfile. If your desired base image is not listed, then visit the full list of images to find an appropriate base image.

Docker File

Create a file named Dockerfile and save it with the following content:

FROM aimle2e/ubuntu-cuda-jupyter:22.04-12.2.2

# Add your packages and/or libraries below
# for example:
# RUN pip install -r requirements.txt

Build the image

$ docker build --platform linux/amd64 -t my-repo/tir-cuda:12.2.2-ubuntu22.04-01 .

Docker File

Create a file named Dockerfile and save it with the following content:

FROM aimle2e/nvidia-pytorch:23.06-2.1.0

# Add your packages and/or libraries below
# for example:
# RUN pip install -r requirements.txt

Build the image

$ docker build --platform linux/amd64 -t my-repo/tir-pytorch:23.06-py3-04 .

Docker File

Create a file named Dockerfile and save it with the following content:

FROM aimle2e/nvidia-tensorflow:23.06-2.12.0

# Add your packages and/or libraries below
# for example:
# RUN pip install -r requirements.txt

Build the image

$ docker build --platform linux/amd64 -t my-repo/tir-tensorflow:23.06-tf2-py3-02

Docker File

Create a file named Dockerfile and save it with the following content:

FROM aimle2e/nvidia-pytorch-transformers:23.06-2.1.0-4.31.0

# Add your packages and/or libraries below
# for example:
# RUN pip install -r requirements.txt

Build the image

$ docker build --platform linux/amd64 -t my-repo/tir-transformers:23.06-py3 .

1.2 Patching an instance of the container and Saving the Image

If you want to skip the hassle of creating and building a Dockerfile, you can modify any of the TIR provided images on you local system and save or commit your changes into a new image. You can then use this new image to launch TIR Notebooks.

Here’s how you can do this:

Step-1: Run a new container from a TIR-provided image

Choose a TIR provided image from this list, as per your requirements and copy its image tag
Create a new container from the selected image using the below command. Replace <image> with the image tag copied in the previous step.
```
$ docker run --platform=linux/amd64 -d <image>
```
If the image is not present in your local system, it will pulled automatically, which can take up a few minutes.
List all your containers using the below command and copy your container’s Container-ID
```
$ docker ps -a
```

Step-2: Modify the image & install the desired libraries

Execute the command below, using the container_id copied in the previous step
```
$ docker exec -it <container_id> /bin/bash
```
This will start an interactive bash shell session within the container. Here, you can install the required libraries and packages just like you would do in a normal shell/terminal.

Once you are done with the changes you need in your custom container, type the exit command to terminate the shell session within the container.
```
$ exit
```

Step-3: Commit your changes into a new image

Now, we want to commit the changes we made and save it as a new image. To do the same, execute the below command. Here, container_id should be the same that you used in Step-2.
```
$ docker commit  <container_id>  <image_repository[:tag]>
```
Here, image_repository and tag is the repository name and tag of your new custom image. tag is optional and defaults to latest

Step-4: Push the custom container image

To use your custom image for TIR Notebooks, you need to push it to E2E Container Registry. Follow the steps in Push the container image subsection below.

1.3 Using Image Builder Utility

The above two methods involves building the custom image by extending the TIR provided images. But if you already have your custom image built and ready, TIR Image Builder Utility tool is what you need.

This tool requires your custom image as an input, makes your custom image compatible with TIR Notebooks and generates a new image that you can use to launch TIR Notebooks. You can make any of your existing images TIR-compatible with the help of this image builder utility.

Steps to use TIR Notebook Image Builder Utility

Step-1: Clone the GitHub repository

Clone the GitHub repository for Notebook Image Builder

$ git clone https://github.com/tire2e/notebook-image-builder.git
$ cd notebook-image-builder/

Step-2: Run image builder utility and generate a new TIR compatible image

Run the command below to generate the TIR-compatible image using the utility tool

$ ./generate_image.sh [OPTIONS]

Below are the details about the supported options that you can pass as arguments to the above command

$ ./generate_image.sh -h

-b,  <string>   (required)    Sets the base image repository (with tag) to be used, including the self-hosted
                              registry path, if any. If the base image is in a self-hosted or private registry,
                              ensure you are logged in.

-i,  <string>   (required)    Sets the name of the custom image repository being built.

-t,  <string>                 Sets the tag for the custom image repository. Defaults to 'latest'.

-P,                           Pushes the image repository to the specified registry

-u,  <string>                 Sets the username to the E2E Container Registry for login.
                              Required if -P flag is set & you aren't logged in to your E2E Container Registry

Example Usage

Generate Image & push it manually later:

To create a TIR-compatible image named tir-my-trainer:v1 with your custom image named my-trainer:latest, run the command below
```
$ ./generate_image.sh -b my-trainer:latest -i tir-my-trainer -t v1
```
You can manually push the image to your project’s container registry by following the steps in Push the container image subsection below.
Generate Image & auto-push it to the registry:

You can also auto-push the image to your container registry by using the -P option in the command, provided your E2E Container Registry is set-up

Let’s say, your namespace for E2E Container Registry is my-project, so the command becomes
```
$ ./generate_image.sh -b my-trainer:latest -i my-project/tir-my-trainer -t v1 -P
```
If you aren’t logged in to the E2E Container Registry, use the -u flag along with -P flag. For example, if your username is abc@xyz.com, then the command would be
```
$ ./generate_image.sh -b my-trainer:latest -i my-project/tir-my-trainer -t v1 -P -u abc@xyz.com
```
When the command execustion completes, the newly created image will be available in the container registry tab of the notebook creation form.

Note

If you haven’t setup a E2E Container Registry yet, follow the steps in the Setup Registry Integration subsection below.

2. Push the container image

Once your custom container image is ready, you need to push it to Container Registry for it to be accessible by TIR. TIR provides several options to work with registry. You can configure multiple registries or let TIR create one for you.

Note

If you have used the image builder utility, and opted for auto-pushing the image to E2E Container Registry, then you don’t need to push the image again.

2.1 Setup Registry Integration

The following steps are required to enable TIR to create and access your project specific container registry:

Go to the TIR Dashboard, choose your target project.

Click Integrations >> E2E Container Registry in Sidebar.

The integrations page allows you to configure third party connections for TIR containers. Click Create E2E Registry.

In the create form, enter your preferred value for Namespace and Username Prefix.

Note

Username Prefix is not your actual username for the registry. Your registry username is denoted against the Username Prefix field.

We advise you to use an easy to remember name as it will be used frequently for docker commands.

Click on create. A new E2E Container Registry will be created for your project.

2.2 Setup Registry Credential on your local system

Let’s now set-up your created E2E Registry on your local system.

Enter the command below on your terminal.

$ docker login registry.e2enetworks.net

It will prompt you to enter your registry username and password. Do the same and press enter to Login.

In case you do not remember your credentials, do not worry. Follow the steps below:

Go to TIR Dashboard >> select your project >> Integrations >> E2E Container registry
Locate your registry from the listing and click on the Commands option.
Copy the Docker Login command and enter it in the terminal to login.

2.3 Push the image

After creating your custom image, you would want to push it to your Registry you just set up.

Tag your local image using the command below

$ docker tag SOURCE_IMAGE[:TAG] registry.e2enetworks.net/NAMESPACE/REPOSITORY[:TAG]
where,

SOURCE_IMAGE is the name of your local image,

NAMESPACE is your E2E Constainer Registry namespace

REPOSITORY is the name of the image repository you want to assign inside your namespace

TAG is the tag of the image repository. It is optional. If it is not specified, it defaults to latest.

Example Usage: To push your local image my_custom_img with tag v1 to your E2E Container Registry with namespace my-cr, command would be like:
$ docker tag my_custom_img:v1 registry.e2enetworks.net/my-cr/my_custom_img:v1

Push the tagged image repository

$ docker push registry.e2enetworks.net/NAMESPACE/REPOSITORY[:TAG]
replace NAMESPACE, REPOSITORY and TAG as specified in the previous step while tagging the image.

Note

Your namespace specific docker commands to tag and push the image is also available on your TIR Dashboard. To access the same:

Go to TIR Dashboard >> select your project >> Integrations >> E2E Container Registry
Locate your registry from the listing and click on the Commands option.
You’ll find your namespace specific docker commands here.

3. Create Notebook

Go to Notebooks in TIR Dashboard
Click CREATE NOTEBOOK button
Enter details as desired except in the images section, choose the option container registry >> select your Container Registry >> select your custom image with tag
Enter rest of the details and launch the notebook
When notebook is ready, you can open the Jupyter labs or use ssh (if opted during creation) to confirm that the packages or libraries from your image are available.