Upload model files

Upload the files of a model to a repository from the portal. When the model is large, upload from your own computer with a tool instead. A repository takes any file, and the files show on its Model Files tab.

Before you begin

• A repository; see Create a repository.
• Permission to update model repositories in the project. A Member gets it through a policy that allows updating for the Inference service.
• Your model files in a folder. A model in TIR is a directory in the bucket, in any structure you like. For an endpoint, lay the folder out the way its framework expects. See Lay out the folder for examples.

Lay out the folder

A PyTorch or Hugging Face model folder looks like this:

my-pytorch-model/
├── config.json
├── pytorch_model.bin
├── tokenizer.json
└── tokenizer_config.json

A Triton repository has a configuration file and a numbered folder that holds the model:

my-triton-model/
├── config.pbtxt
└── 1/
    └── model.graphdef

A custom model can be any layout, such as this one:

my-custom-model/
├── model.onnx
└── config.yaml

Upload from the portal

1. Select the row of the repository, open the Model Files tab, and select Upload.
2. Destination Bucket / Folder Path starts as /. Leave it to put the files at the top of the bucket, or type a folder such as v1 or production.
3. Choose or drag in up to 10 files. For a whole folder, use the MinIO CLI below.
4. Select Upload, and follow each file’s progress.
5. When every file has finished, select Ok or close the panel to reload the file list.

The panel recommends the MinIO CLI or s3cmd for files over 5 GB. Adding an 11th file brings up “Only 10 files can be uploaded at once”.

Upload a folder with the MinIO CLI or s3cmd

Connect the tool first; see Connect to a repository with the MinIO CLI or s3cmd. Then copy the folder. mc cp -r keeps the folder layout in the bucket. A folder after the bucket name, such as v1 or production, makes a version.

mc cp -r /path/to/model/* <repository-name>/<bucket-name>/
mc cp -r /path/to/model/* <repository-name>/<bucket-name>/v1/

To upload a Hugging Face snapshot, copy the downloaded folder. <model-name> is the name of the model folder in your Hugging Face cache:

mc cp -r ~/.cache/huggingface/hub/<model-name>/* <repository-name>/<bucket-name>/

To copy several files at once, use mc cp with --max-workers. It has no --parallel option.

Upload with the Python SDK

The Using SDK tab gives the code with your project and repository numbers filled in.

1. Open the Using SDK tab. Choose a token in Select Token, or select Generate token to make one. The tab asks for it with “Before you proceed, you will need an API token to access the TIR SDK.”

2. Install the SDK:

   pip install e2enetworks

3. Run the code in a Python shell or a Jupyter notebook. Take <api-key> and <access-token> from Projects > API Tokens. <project-id> is your project’s number, and <repository-id> is the repository’s number. The tab fills in those last two.

   from e2enetworks.cloud import tir
   from e2enetworks.constants import CHENNAI_LOCATION, DELHI_LOCATION
   
   location = DELHI_LOCATION  # or CHENNAI_LOCATION
   tir.init(api_key="<api-key>", access_token="<access-token>", location=location)
   
   models = tir.Models(project=<project-id>)
   models.push_model(model_path="./model-dir", prefix="", model_id=<repository-id>)
   models.push_model(model_path="./model-dir", prefix="v1", model_id=<repository-id>)

   The first call uploads the folder to the top of the bucket. The second uploads it as a version in v1. Set location to the region you are working in.

Keep versions in folders

Versions are folders in the bucket, and you work with them like any other folder. Use /v1 and /v2 for numbered versions, /production and /staging for environments, and /experiments/exp-001 for experiments. When you link an endpoint, you give the folder as the model path. Keep the two or three previous versions for rollback, and archive or delete older ones to save storage.

Check the upload

Open the Model Files tab and look for the files. A new repository starts empty. If a file is missing, see Troubleshooting Model Repository.

To open the files, delete some, or download one, see Browse, download, and delete files.