Create Model Evaluation

POST

/model_evaluation/

curl --request POST \
  --url 'https://api.e2enetworks.com/myaccount/api/v1/gpu/model_evaluation/?project_id=<PROJECT_ID>&location=<LOCATION>&apikey=<API_KEY>' \
  --header 'Authorization: Bearer <API_TOKEN>' \
  --header 'Content-Type: application/json' \
  --data '{
  "name": "tir-job-021915025656",
  "input_dataset_id": "",
  "output_dataset_id": "async-invocations-02171142022",
  "input_dataset_config": {
    "dataset_type": "huggingface",
    "dataset_split": "train",
    "input_column": "question",
    "output_column": "answer",
    "reference_answer_column": "",
    "num_rows_limit": -1,
    "custom_hf_dataset_name": "",
    "hf_dataset_name": "GAIR/o1-journey"
  },
  "framework_id": 1,
  "evaluator_llm_id": 2,
  "evaluation_config": {
    "temperature": 0,
    "max_tokens": 1024,
    "top_p": 1
  },
  "model_evaluation_prompt": "test",
  "sku_item_price_id": 207,
  "huggingface_integration_id": null
}'

Parameters

Query Parameters

project_id

required

integer

Project ID

active_iam

integer

Active IAM ID (To access contact person account) Find your Active IAM ID here

location

required

string

Allowed values: Delhi Chennai

Location

Request Body^required

Media typeapplication/json

object

name

string

Example

tir-job-021915025656

input_dataset_id

string

Example

output_dataset_id

string

Example

async-invocations-02171142022

input_dataset_config

object

dataset_type

string

Allowed values: huggingface EOS Dataset

Example

huggingface

dataset_split

string

Example

train

input_column

string

Example

question

output_column

string

Example

answer

reference_answer_column

string

Example

num_rows_limit

integer

Example

-1

custom_hf_dataset_name

string

Example

hf_dataset_name

string

Allowed values: GAIR/o1-journey chenle015/OpenMP_Question_Answering sujayC66/text_summarization_512_length_1_2000 JeswinMS4/code_text_classification

Example

GAIR/o1-journey

framework_id

1 - text-summarization 2 - general_assistant 3 - question_answering 4 - text_classification

integer

Example

1

evaluator_llm_id

1 - llama3_2_3b_instruct 2 - llama3_1_8b_instruct

integer

Example

2

evaluation_config

object

temperature

number format: float

<= 1

Example

0

max_tokens

integer

>= 16 <= 2000

Example

1024

top_p

number format: float

<= 1

Example

1

model_evaluation_prompt

string

Example

test

sku_item_price_id

SKU item price ID for the evaluation job. This value is region-specific. Use the plans-and-pricing API to get valid IDs for your region.

integer

Example

207

huggingface_integration_id

string

nullable

Example

null

Responses

201

Evaluation request created successfully

Media typeapplication/json

object

code

integer

data

object

id

integer

name

string

project_id

integer

input_dataset

string

nullable

input_dataset_config

object

dataset_path

string

nullable

hf_dataset_name

string

custom_hf_dataset_name

string

dataset_split

string

custom_dataset_split

string

nullable

input_column

string

output_column

string

reference_answer_column

string

num_rows_limit

integer

dataset_type

string

output_dataset

object

id

integer

name

string

project_id

integer

description

string

nullable

storage_type

string

pvc

string

nullable

bucket

object

id

integer

bucket_name

string

bucket_url

string

endpoint

string

is_managed

boolean

eos_bucket_permission_id

integer

status

string

framework

object

id

integer

name

string

display_name

string

description

string

evaluator_llm_name

string

status

string

created_at

string format: date-time

errors

object

message

string

Example

{
  "code": 201,
  "data": {
    "id": 126,
    "name": "tir-job-021915025656",
    "project_id": 4528,
    "input_dataset": null,
    "input_dataset_config": {
      "dataset_path": null,
      "hf_dataset_name": "GAIR/o1-journey",
      "custom_hf_dataset_name": "",
      "dataset_split": "train",
      "custom_dataset_split": null,
      "input_column": "question",
      "output_column": "answer",
      "reference_answer_column": "",
      "num_rows_limit": -1,
      "dataset_type": "huggingface"
    },
    "output_dataset": {
      "id": 1617,
      "name": "async-invocations-02171142022",
      "project_id": 4528,
      "description": null,
      "storage_type": "managed",
      "pvc": null,
      "bucket": {
        "id": 3068,
        "bucket_name": "d-gpu-6a93eb6a-5a3e-411e-80ec-b91b31e7a93f",
        "bucket_url": "s3://d-gpu-6a93eb6a-5a3e-411e-80ec-b91b31e7a93f/",
        "endpoint": "https://objectstore.e2enetworks.net",
        "is_managed": true,
        "eos_bucket_permission_id": 11488
      },
      "status": "OK"
    },
    "framework": {
      "id": 2,
      "name": "general_assistant",
      "display_name": "General Assistant",
      "description": "Use this metric for evaluating the general assistant capability of your LLM."
    },
    "evaluator_llm_name": "llama3_2_3b_instruct",
    "status": "Creating",
    "created_at": "2025-02-19T09:34:52.615513Z"
  },
  "errors": {},
  "message": "Created Successfully"
}