--- title: "Deploy a custom container" description: "Deploy your own Docker image as a Model Endpoint with the Custom Container framework, set its ports, start command, and health checks, and call it." content_type: "how-to" product: "model-endpoints" platform: "TIR" url: "/tir/model-endpoints/how-to/deploy-a-custom-container/" topic: ["create"] --- # Deploy a custom container Run your own Docker image behind a Model Endpoint when your model or your serving code needs more than the other frameworks offer. The endpoint runs your image on the machine you choose and gives it an HTTPS address. ## Before you begin - Permission to create model endpoints in the project. A Member needs a policy that grants create permission for the Inference service. - An image that serves your model on a port, and its address. A private image needs a registry; see [Container Registry](/tir/tir-container-registry/) for E2E Registry. - Model files, if the container downloads them. They can come from a [Model Repository](/tir/model-repository/) or from a GitHub repository. ## Choose Custom Container 1. On **Choose Framework**, select **Custom Container**. See [Choose a framework](/tir/model-endpoints/how-to/choose-a-framework/). 2. In **Download Type**, choose what the container needs. It starts on **Skip Model Download**, which skips the model download at startup. **Link with Model Repository** downloads the model from the repository you choose when the container starts or restarts; see [Deploy a model from Model Repository](/tir/model-endpoints/how-to/deploy-a-model-from-model-repository/). **Link with Github** downloads the model from GitHub. ## Set the container In **Container Configuration**, choose where the image lives, then fill in the rest. 1. Choose **Public** or **Private**. 2. For a public image, type the **Image**, up to 128 characters, for example `nginx:latest`. 3. For a private image, choose a **Registry Type**. With **Nvidia NGC Registry**, type the image address. With **E2E Registry**, choose a registry namespace, an image, and a tag. Refresh the list if it is out of date, or follow the link to create a registry. 4. Set **Image Pull Policy**. **Always** is the default, and **If not present** pulls the image only when it is missing from the machine. 5. Set **Service Type**. **External** is reachable from the internet through the root endpoint with a token. **Internal** is reachable only from inside the project's network. 6. Under **Application Port**, tick **HTTP**, **gRPC**, or both. On an **External** endpoint, HTTP uses port 8080 and gRPC uses port 9000. Your container must listen on those ports. On an **Internal** endpoint you can add several HTTP ports: type a port and press Enter or a comma. TIR refuses ports 80 and 8082 as internal HTTP ports. While gRPC is on, port 9000 is reserved for gRPC. 7. Set **Metrics Port**, from 1 to 65535, where your application serves metrics at `/metrics`. It starts at 8080. ## Set the start command and health checks **Advanced Configuration** holds the start command and two health checks. - Set the start command in three parts. Choose a **Serving Framework**. **vLLM**, **SGLang**, and **Triton Inference Server** fill in that server's usual command and arguments, with the port set to 8080. **Bash** takes your whole command as one shell line, **Custom** starts empty for your own command, and **SGLang Multimodal** is on the list too. Then check the **Entrypoint Command** and add **Arguments** as rows of **Flag** and **Value** with **Add argument**. Check the whole command under **Command Preview**, where you can also copy it. - Turn on **Readiness Probe** and **Liveness Probe** if you want them. Each offers three protocols. **HTTP** takes a **URL** path. **Execute** takes a **Commands** list in square brackets, such as `["cat", "tmp/healthy"]`. **TCP** takes a **Port** from 1 to 65535. A probe you turn on starts with **Period Seconds** 10, **Timeout Seconds** 10, **Failure Threshold** 3, **Initial Delay Seconds** 10, and **Success Threshold** 1, with the readiness path `/health`, the liveness path `/metrics`, and port 8080. Only a probe you turn on is applied. ## (Optional) Add environment variables Open **Advanced Settings** to pass values your container expects. See [Set environment variables](/tir/model-endpoints/how-to/set-environment-variables/). ## (Optional) Redeploy from GitHub With **Link with Github**, type the **GitHub Repository** as `https://github.com///tree/` and choose a **GitHub Integration** that can reach it. The form says "GitHub access token has access to the repository." when the check passes. See [Create a GitHub integration](/tir/integrations/how-to/create-a-github-integration/). Before you deploy, add the workflow file at `.github/workflows/deploy.yml` in your repository. The form offers the file for download. After the endpoint exists, its **Integrations** tab has **Download yml**, the endpoint's UUID to paste into the workflow, and the two repository secrets to add. When someone merges a change, the workflow restarts the endpoint so that it pulls the new code. **Download YML** is also in the **Actions** menu. ## Deploy and call it Choose the machine and workers as in [Create an endpoint](/tir/model-endpoints/how-to/create-an-endpoint/), then select **Deploy Endpoint**. For an external endpoint, the **cURL** sample is a request to the root endpoint and a route that you fill in, with your own body. See [Call an endpoint with a token](/tir/model-endpoints/how-to/call-an-endpoint-with-a-token/). An internal endpoint needs no token and is called at `http://is-.p-:`, where the port is the first HTTP port, or 9000 for gRPC only. ## Change the container later To change the image, ports, or commands on a running endpoint, open **Actions** and select **Update Container Configuration**. You change the same container settings as at create, except the name and framework. ## Check the container The endpoint becomes **Running** when its workers are ready. If a worker shows **ImagePullBackOff** or **ErrImagePull**, the worker failed to fetch the image. Check the image name, tag, and registry login. **CrashLoopBackOff** means the container keeps starting and stopping, so read **Worker Logs**. See [View logs and events](/tir/model-endpoints/how-to/view-logs-and-events/) and [Troubleshooting Model Endpoints](/tir/model-endpoints/troubleshooting/model-endpoints/).