Update/Actions (Start/Stop/Restart/Autoscale/Async)

PUT

/serving/inference/{inference_id}/

curl --request PUT \
  --url 'https://api.e2enetworks.com/myaccount/api/v1/gpu/serving/inference/{inference_id}/?project_id=<PROJECT_ID>&location=<LOCATION>&apikey=<API_KEY>' \
  --header 'Authorization: Bearer <API_TOKEN>' \
  --header 'Content-Type: application/json' \
  --data '{
  "name": "string",
  "path": "string",
  "custom_endpoint_details": {
    "service_port": false,
    "metric_port": false,
    "container": {
      "advance_config": {
        "liveness_probe": {
          "commands": "string"
        },
        "readiness_probe": {
          "path": "string",
          "port": "string",
          "protocol": "string",
          "period_seconds": 0,
          "timeout_seconds": 0,
          "failure_threshold": 0,
          "success_threshold": 0,
          "initial_delay_seconds": 0,
          "commands": "string"
        },
        "image_pull_policy": "string",
        "is_liveness_probe_enabled": false,
        "is_readiness_probe_enabled": false
      },
      "container_name": "string",
      "container_type": "string",
      "private_image_details": {}
    },
    "public_ip": "string",
    "resource_details": {
      "disk_size": 0,
      "mount_path": "string",
      "env_variables": [
        "string"
      ]
    }
  },
  "model_id": "string",
  "committed_replicas": 0,
  "replica": 0,
  "framework": "string",
  "is_auto_scale_enabled": false,
  "auto_scale_policy": {
    "min_replicas": 0,
    "max_replicas": 0,
    "rules": [
      {
        "value": 0,
        "metric": "string",
        "window": 0,
        "granularity": 0,
        "watch_period": 0,
        "condition_type": "string",
        "custom_metric_name": "string"
      }
    ],
    "stability_period": "string",
    "initial_cooldown_period": "string",
    "is_hold_queue": false
  },
  "detailed_info": {
    "args": "string",
    "commands": "string",
    "info_log": false,
    "error_log": false,
    "tokenizer": "string",
    "world_size": 0,
    "engine_args": {},
    "warning_log": false,
    "server_version": "string",
    "hugging_face_id": "string",
    "model_serve_type": "string",
    "log_verbose_level": 0
  },
  "model_load_integration_id": 0,
  "dataset_id": "string",
  "dataset_path": "string",
  "cluster_type": "string",
  "storage_type": "string",
  "sfs_path": "string",
  "disk_path": "string",
  "sku_id": 0,
  "sku_item_price_id": 0,
  "action": "string",
  "async_enabled": false,
  "async_concurrent_workers": 0,
  "async_dataset_id": 0,
  "async_destination_type": "string",
  "async_target_routes": [
    {
      "route_name": "string",
      "route_value": "string"
    }
  ]
}'

Use PUT to update model endpoint configuration or perform actions.

action value	Purpose
patch	Update endpoint configuration
restart	Restart the running endpoint
stop	Stop the endpoint
start	Start a stopped endpoint
update_auto_scale	Create or update the autoscaling policy
update_async_configuration	Enable or disable async invocation

Autoscaling metrics

The metric field in each rule supports:

• concurrency — scales based on the number of concurrent in-flight requests.
• requestRate — scales based on incoming requests per second.
• custom — scales based on a framework-specific metric; requires custom_metric_name.

Async Invocation

Use action: "update_async_configuration" to enable or disable async invocation.

async_enabled	Effect
true	Enables async invocation with the specified worker count and dataset destination
false	Disables async invocation

When enabling, provide:

• async_concurrent_workers — number of parallel async workers
• async_dataset_id — ID of the dataset to store async results
• async_destination_type — destination type (e.g. "dataset")
• async_target_routes — list of route mappings (route_name → route_value)

Parameters

Path Parameters

inference_id

required

integer

Query Parameters

project_id

required

integer

Project ID

active_iam

integer

Active IAM ID (To access contact person account) Find your Active IAM ID here

location

required

string

Allowed values: Delhi Chennai

Location

Request Body

Media typeapplication/json

object

name

The name of the model configuration.

string

path

The path for the model configuration. Can be empty.

string

custom_endpoint_details

object

service_port

Whether a service port is enabled.

boolean

metric_port

Whether a metric port is enabled.

boolean

container

object

advance_config

object

liveness_probe

object

commands

Liveness probe commands.

string

readiness_probe

object

path

The readiness probe path.

string

port

The port for readiness probe.

string

protocol

The protocol for readiness probe.

string

period_seconds

Period in seconds to check readiness.

integer

timeout_seconds

Timeout for readiness probe.

integer

failure_threshold

Failure threshold for readiness probe.

integer

success_threshold

Success threshold for readiness probe.

integer

initial_delay_seconds

Initial delay before readiness probe starts.

integer

commands

Readiness probe commands.

string

image_pull_policy

The image pull policy.

string

is_liveness_probe_enabled

Whether the liveness probe is enabled.

boolean

is_readiness_probe_enabled

Whether the readiness probe is enabled.

boolean

container_name

The container image name.

string

container_type

The type of the container (e.g., public or private).

string

private_image_details

Details about the private container image, if applicable.

object

public_ip

Whether a public IP is used (yes/no).

string

resource_details

object

disk_size

Disk size in GB.

integer

mount_path

Path to mount for resources.

string

env_variables

List of environment variables for the container.

Array<string>

model_id

The model ID (null if not specified).

string

committed_replicas

Number of committed replicas.

integer

replica

Number of replicas to run.

integer

framework

The framework used (e.g., vllm).

string

is_auto_scale_enabled

Whether auto-scaling is enabled.

boolean

auto_scale_policy

object

min_replicas

Minimum number of replicas for auto-scaling.

integer

max_replicas

Maximum number of replicas for auto-scaling.

integer

rules

Array<object>

object

value

Metric value for auto-scaling rule.

integer

metric

Metric to track for auto-scaling.

string

window

The window for metric evaluation.

integer

granularity

Granularity for evaluation.

integer

watch_period

Watch period for metric evaluation.

integer

condition_type

The condition type for scaling.

string

custom_metric_name

The custom metric name if applicable.

string

stability_period

Stability period (in seconds) — minimum time the system waits after a scale event before evaluating again.

string

initial_cooldown_period

Cooldown period (in seconds) applied after the very first scale-up before further scaling decisions are evaluated.

string

is_hold_queue

When true, incoming requests are held in a queue while all replicas are busy. When false, excess requests are rejected immediately.

boolean

detailed_info

object

args

Arguments passed to the model.

string

commands

Commands executed for the model.

string

info_log

Whether info logging is enabled.

boolean

error_log

Whether error logging is enabled.

boolean

tokenizer

Tokenizer to use for model processing.

string

world_size

The world size for distributed processing.

integer

engine_args

Additional arguments for the model engine.

object

warning_log

Whether warning logging is enabled.

boolean

server_version

The server version for deployment.

string

hugging_face_id

The hugging face ID for the model.

string

model_serve_type

Type of model serving method.

string

log_verbose_level

Log verbosity level.

integer

model_load_integration_id

Integration ID for model load.

integer

dataset_id

Dataset ID associated with the model.

string

dataset_path

Path to the dataset if available.

string

cluster_type

The type of cluster for deployment.

string

storage_type

The storage type used (e.g., disk).

string

sfs_path

Path for the shared file system.

string

disk_path

Path to the disk storage.

string

sku_id

SKU ID for the model.

integer

sku_item_price_id

Price ID for the model.

integer

action

The action to be performed (e.g., patch, restart, stop, start, update_auto_scale, update_async_configuration).

string

async_enabled

Set true to enable async invocation, false to disable. Used with action: update_async_configuration.

boolean

async_concurrent_workers

Number of parallel async workers. Required when enabling async invocation.

integer

async_dataset_id

ID of the dataset to store async results. Required when enabling async invocation.

integer

nullable

async_destination_type

Destination type for async results (e.g. "dataset").

string

nullable

async_target_routes

List of route mappings for async inference.

Array<object>

object

route_name

Internal queue name (e.g. "QUEUE_1").

string

route_value

Endpoint route to call (e.g. "predict").

string

Example ^generated

{
  "name": "example",
  "path": "example",
  "custom_endpoint_details": {
    "service_port": true,
    "metric_port": true,
    "container": {
      "advance_config": {
        "liveness_probe": {
          "commands": "example"
        },
        "readiness_probe": {
          "path": "example",
          "port": "example",
          "protocol": "example",
          "period_seconds": 1,
          "timeout_seconds": 1,
          "failure_threshold": 1,
          "success_threshold": 1,
          "initial_delay_seconds": 1,
          "commands": "example"
        },
        "image_pull_policy": "example",
        "is_liveness_probe_enabled": true,
        "is_readiness_probe_enabled": true
      },
      "container_name": "example",
      "container_type": "example",
      "private_image_details": {}
    },
    "public_ip": "example",
    "resource_details": {
      "disk_size": 1,
      "mount_path": "example",
      "env_variables": [
        "example"
      ]
    }
  },
  "model_id": "example",
  "committed_replicas": 1,
  "replica": 1,
  "framework": "example",
  "is_auto_scale_enabled": true,
  "auto_scale_policy": {
    "min_replicas": 1,
    "max_replicas": 1,
    "rules": [
      {
        "value": 1,
        "metric": "example",
        "window": 1,
        "granularity": 1,
        "watch_period": 1,
        "condition_type": "example",
        "custom_metric_name": "example"
      }
    ],
    "stability_period": "example",
    "initial_cooldown_period": "example",
    "is_hold_queue": true
  },
  "detailed_info": {
    "args": "example",
    "commands": "example",
    "info_log": true,
    "error_log": true,
    "tokenizer": "example",
    "world_size": 1,
    "engine_args": {},
    "warning_log": true,
    "server_version": "example",
    "hugging_face_id": "example",
    "model_serve_type": "example",
    "log_verbose_level": 1
  },
  "model_load_integration_id": 1,
  "dataset_id": "example",
  "dataset_path": "example",
  "cluster_type": "example",
  "storage_type": "example",
  "sfs_path": "example",
  "disk_path": "example",
  "sku_id": 1,
  "sku_item_price_id": 1,
  "action": "example",
  "async_enabled": true,
  "async_concurrent_workers": 1,
  "async_dataset_id": 1,
  "async_destination_type": "example",
  "async_target_routes": [
    {
      "route_name": "example",
      "route_value": "example"
    }
  ]
}

Responses

200

Action performed successfully on the model endpoint.

Media typeapplication/json

object

code

HTTP status code.

integer

data

Message describing the result of the action performed.

string

errors

object

message

string

Example

{
  "code": 200,
  "data": "Model Endpoint restarted successfully",
  "message": "Success"
}