Skip to main content

Deploy Model Endpoint for YOLOv8

YOLOv8 is the latest model in the YOLO (You Only Look Once) series — a highly efficient family of models used for object detection and classification in computer vision tasks.

Overview​

This tutorial will cover the following:

We’ll use the pre-built YOLOv8 container for this tutorial. You can also build a custom container using Custom Inference if needed.

Using the pre-built container eliminates the need to create a custom API handler — platform automatically generates one for you.

A Guide on Model Endpoint Creation and Video Processing Using YOLOv8​

Step 1: Create a Model Endpoint for YOLOv8​

  1. Go to the AI Platform.
  2. Choose your project.
  3. Open the Model Endpoints section.
  4. Click Create Endpoint.
  5. Select YOLOv8 in the framework list.
  6. Choose an appropriate GPU or CPU plan.
  7. Optionally, set environment variables.
  8. In Model Details, skip the custom model selection to use default weights.
  9. Click Create to deploy the endpoint.

If you wish to use your own trained weights, refer to Creating a Model Endpoint with custom weights.

Step 2: Generate Your API Token​

Each model endpoint requires a valid Auth Token for API access.

  1. Navigate to API Tokens in your project.
  2. Create a new API Token or use an existing one.
  3. Copy the Auth Token value for later use.

Step 3: Perform Object Detection on a Video​

Prerequisites​

  1. Create a bucket in E2E Object Storage.
  2. Upload your input video file to that bucket.
  3. Ensure your model endpoint is in Ready state.

Then, get the Sample API Request from the endpoint page.

Below is a sample Python script for processing a video:

Click to expand code
import requests  

# Enter your auth token here
auth_token = "your-auth-token"

url = "https://jupyterlabs.e2enetworks.net/project/p-<project-id>/endpoint/is-<inference-id>/predict"

payload = {
    "input": "$INPUT_VIDEO_FILE_NAME",  # Path to input video in the bucket
    "bucket_name": "$BUCKET_NAME",      # Bucket name
    "access_key": "$ACCESS_KEY",        # Access key of the bucket
    "secret_key": "$SECRET_KEY"         # Secret key of the bucket
}

headers = {
    'Content-Type': 'application/json',
    'Authorization': f'Bearer {auth_token}'
}

response = requests.post(url, json=payload, headers=headers)
print(response.content)  # Returns JOB_ID for tracking video processing

You’ll receive a JOB_ID in the response. Use it to track the video processing status.

Step 4: Track Video Processing Status​

Use the following script to check job status:

Click to expand code
import requests  

auth_token = "your-auth-token"
url = "https://jupyterlabs.e2enetworks.net/project/p-<project-id>/endpoint/is-<inference-id>/status/$JOB_ID"

headers = {
    'Content-Type': 'application/json',
    'Authorization': f'Bearer {auth_token}'
}

response = requests.get(url, headers=headers)
print(response.content)

Ensure the following before running:

  • Correct Auth Token is provided.
  • Correct bucket name, access key, secret key, and job ID are specified.
Note

Video output files are in .avi format. Only .avi outputs are supported currently.

Once complete, your processed video will appear in the bucket under yolov8/outputs/.

Creating Model Endpoint with Custom Model Weights​

You can deploy your own fine-tuned YOLOv8 weights by following these steps:

Step 1: Define a Model in Dashboard​

  1. Go to AI Platform.
  2. Choose a project.
  3. Open the Model section.
  4. Click Create Model.
  5. Name your model (e.g., yolov8-custom).
  6. Select Model Type as Custom.
  7. Click CREATE.
  8. Copy the Setup Host command from the Setup MinIO CLI tab.
Note

If you forget to copy the Setup Host command, you can revisit the model’s detail page to retrieve it.

Step 2: Upload the Model to EOS (E2E Object Storage)​

Use the MinIO CLI to upload your model file.

# Go to Dashboard → Models → Select your model → Setup MinIO CLI tab.

# Example copy command:
mc cp -r <MODEL_NAME>.pt yolov8/yolov8-custom-854588

Ensure the uploaded file is in .pt (PyTorch) format — other formats are not supported.

To list directories before uploading:

$HOME/.cache/huggingface/hub

Step 3: Create an Endpoint for Custom Model​

Follow the same steps from Model Endpoint creation , but select your uploaded custom model under Model Details.

If the model is not in the root bucket directory, specify its path in the Model Path field during endpoint creation.

Ensure the .pt extension is correct and the file path is valid.

Your YOLOv8 model endpoint is now deployed and ready for inference using either pre-trained or custom-trained weights!