Scheduled Run

A “Scheduled Run” refers to the automation of the pipeline to execute at specific predetermined times or intervals. This scheduling helps ensure regular and timely execution of tasks within the pipeline, making it useful for various purposes, such as data processing, model training, or deployment. You can set up a scheduled run for a specific date and time. After that, the designated pipeline will automatically run at the scheduled date and time.

List of Version

To get the list of Version, send a GET request to the Pipeline Run Endpoint

https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/{{pipeline_id}}/versions/?apikey={{tapi_key}}

PythonCurl

import requests

url = "https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/{{pipeline_id}}/versions/?apikey={{tapi_key}}"

payload={}
headers = {
'Authorization': 'Bearer {{Token}}',
'Content-Type': 'application/json',
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Request HeadersResponse Headers

Content-Type: application/json
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAi...

List of Experiments

To get the list of Experiments, send a GET request to the Pipeline Run Endpoint

https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/experiments?apikey={{tapi_key}}

PythonCurl

import requests

url = "https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/experiments?apikey={{tapi_key}}"

payload={}
headers = {
'Authorization': 'Bearer {{Token}}',
'Content-Type': 'application/json',
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Request HeadersResponse Headers

Content-Type: application/json
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAi...

List of SKU

To get the list of SKU, send a GET request to the Pipeline Run Endpoint

https://api.e2enetworks.com/myaccount/api/v1/gpu/gpu_service/sku/?apikey={{tapi_key}}&image_id=&service=pipeline

PythonCurl

import requests

url = "https://api.e2enetworks.com/myaccount/api/v1/gpu/gpu_service/sku/?apikey={{tapi_key}}&image_id=&service=pipeline"

payload={}
headers = {
'Authorization': 'Bearer {{Token}}',
'Content-Type': 'application/json',
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Request HeadersResponse Headers

Content-Type: application/json
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAi...

Run Scheduled Jobs

To run the Scheduled jobs, send a POST request to the Fine Tuning Scheduled Run Endpoint

https://api.e2enetworks.com/myaccount/api/v1/sgpu/projects/{{project_id}}/pipelines/jobs/?apikey={{tapi_key}}

PythonCurl

import requests
import json

url = "https://api.e2enetworks.com/myaccount/api/v1/sgpu/projects/{{project_id}}/pipelines/jobs/?apikey={{tapi_key}}"

payload = json.dumps({
"sku_id": 81,
"sku_item_price_id": 207,
"next_sku_item_price_id": None,
"committed_instance_policy": "",
"name": "tir-run-070819204242",
"description": "",
"experiment_id": "{{experiment_id}}",
"pipeline_type": "existing",
"pipeline_version_id": "{{version_id}}",
"run_parameter": [
    {
    "name": "workspace_pvc_class",
    "value": "csi-rbd-sc"
    },
    {
    "name": "hf_api_token",
    "value": ""
    },
    {
    "name": "e2e_tir_access_token",
    "value": ""
    },
    {
    "name": "e2e_tir_api_key",
    "value": ""
    },
    {
    "name": "e2e_tir_project_id",
    "value": ""
    },
    {
    "name": "e2e_tir_team_id",
    "value": ""
    },
    {
    "name": "e2e_tir_finetune_job_id",
    "value": ""
    },
    {
    "name": "e2e_tir_api_host",
    "value": ""
    },
    {
    "name": "run_name",
    "value": ""
    },
    {
    "name": "source_model_repo_id",
    "value": "0"
    },
    {
    "name": "source_model_path",
    "value": ""
    },
    {
    "name": "model",
    "value": "google/gemma-7b-it"
    },
    {
    "name": "dataset_name",
    "value": "luisroque/instruct-python-llama2-20k"
    },
    {
    "name": "dataset_text_field",
    "value": "text"
    },
    {
    "name": "dataset_type",
    "value": "huggingface"
    },
    {
    "name": "dataset_bucket",
    "value": ""
    },
    {
    "name": "dataset_path",
    "value": ""
    },
    {
    "name": "dataset_accesskey",
    "value": ""
    },
    {
    "name": "dataset_secretkey",
    "value": ""
    },
    {
    "name": "dataset_split",
    "value": "1"
    },
    {
    "name": "batch_size",
    "value": "1"
    },
    {
    "name": "gradient_accumulation_steps",
    "value": "1"
    },
    {
    "name": "seq_length",
    "value": "512"
    },
    {
    "name": "num_train_epochs",
    "value": "3"
    },
    {
    "name": "max_steps",
    "value": "-1"
    },
    {
    "name": "save_steps",
    "value": "100"
    },
    {
    "name": "save_total_limit",
    "value": "10"
    },
    {
    "name": "use_peft",
    "value": "true"
    },
    {
    "name": "peft_lora_r",
    "value": "64"
    },
    {
    "name": "peft_lora_alpha",
    "value": "16"
    },
    {
    "name": "lora_dropout",
    "value": "0.05"
    },
    {
    "name": "lora_bias",
    "value": "none"
    },
    {
    "name": "load_in_4bit",
    "value": "false"
    },
    {
    "name": "bnb_4bit_compute_dtype",
    "value": "bfloat16"
    },
    {
    "name": "bnb_4bit_quant_type",
    "value": "fp4"
    },
    {
    "name": "bnb_4bit_use_double_quant",
    "value": "false"
    },
    {
    "name": "trust_remote_code",
    "value": "false"
    },
    {
    "name": "use_auth_token",
    "value": "true"
    },
    {
    "name": "trainer_ram",
    "value": "100"
    },
    {
    "name": "trainer_gpus",
    "value": "1"
    },
    {
    "name": "trainer_cores",
    "value": "8"
    },
    {
    "name": "ephemeral_storage",
    "value": "99"
    },
    {
    "name": "finetuner_image",
    "value": "aimle2e/fine-tuner"
    },
    {
    "name": "finetuner_tag",
    "value": "gemma_7b_inst_v5"
    },
    {
    "name": "finetuner_image_policy",
    "value": "Always"
    },
    {
    "name": "sku_type",
    "value": ""
    },
    {
    "name": "wandb_key",
    "value": ""
    },
    {
    "name": "wandb_project",
    "value": ""
    },
    {
    "name": "wandb_run_name",
    "value": ""
    },
    {
    "name": "learning_rate",
    "value": "0.00002"
    },
    {
    "name": "max_train_samples",
    "value": "10"
    },
    {
    "name": "max_eval_samples",
    "value": "10"
    },
    {
    "name": "workspace",
    "value": "/mnt/workspace"
    },
    {
    "name": "torch_cache",
    "value": "/mnt/workspace/torch"
    },
    {
    "name": "transformers_cache",
    "value": "/mnt/workspace/hf"
    },
    {
    "name": "output_dir",
    "value": "/mnt/workspace/finetunes/"
    },
    {
    "name": "prompt_template_base64",
    "value": ""
    },
    {
    "name": "resume",
    "value": "true"
    }
],
"run_type": "scheduled_job",
"periodic_schedule": {
    "interval_second": 86400,
    "start_time": "2024-07-08T18:25:42.652652Z",
    "end_time": "2024-07-09T18:25:42.653653Z"
}
})
headers = {
'Authorization': 'Bearer {{Token}}',
'Content-Type': 'application/json',
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Request HeadersResponse Headers

Content-Type: application/json
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAi...

Request Body

{
"sku_id": 81,
"sku_item_price_id": 207,
"next_sku_item_price_id": null,
"committed_instance_policy": "",
"name": "tir-run-070819204242",
"description": "",
"experiment_id": "{{experiment_id}}",
"pipeline_type": "existing",
"pipeline_version_id": "{{version_id}}",
"run_parameter": [
    {
        "name": "workspace_pvc_class",
        "value": "csi-rbd-sc"
    },
    {
        "name": "hf_api_token",
        "value": ""
    },
    {
        "name": "e2e_tir_access_token",
        "value": ""
    },
    {
        "name": "e2e_tir_api_key",
        "value": ""
    },
    {
        "name": "e2e_tir_project_id",
        "value": ""
    },
    {
        "name": "e2e_tir_team_id",
        "value": ""
    },
    {
        "name": "e2e_tir_finetune_job_id",
        "value": ""
    },
    {
        "name": "e2e_tir_api_host",
        "value": ""
    },
    {
        "name": "run_name",
        "value": ""
    },
    {
        "name": "source_model_repo_id",
        "value": "0"
    },
    {
        "name": "source_model_path",
        "value": ""
    },
    {
        "name": "model",
        "value": "google/gemma-7b-it"
    },
    {
        "name": "dataset_name",
        "value": "luisroque/instruct-python-llama2-20k"
    },
    {
        "name": "dataset_text_field",
        "value": "text"
    },
    {
        "name": "dataset_type",
        "value": "huggingface"
    },
    {
        "name": "dataset_bucket",
        "value": ""
    },
    {
        "name": "dataset_path",
        "value": ""
    },
    {
        "name": "dataset_accesskey",
        "value": ""
    },
    {
        "name": "dataset_secretkey",
        "value": ""
    },
    {
        "name": "dataset_split",
        "value": "1"
    },
    {
        "name": "batch_size",
        "value": "1"
    },
    {
        "name": "gradient_accumulation_steps",
        "value": "1"
    },
    {
        "name": "seq_length",
        "value": "512"
    },
    {
        "name": "num_train_epochs",
        "value": "3"
    },
    {
        "name": "max_steps",
        "value": "-1"
    },
    {
        "name": "save_steps",
        "value": "100"
    },
    {
        "name": "save_total_limit",
        "value": "10"
    },
    {
        "name": "use_peft",
        "value": "true"
    },
    {
        "name": "peft_lora_r",
        "value": "64"
    },
    {
        "name": "peft_lora_alpha",
        "value": "16"
    },
    {
        "name": "lora_dropout",
        "value": "0.05"
    },
    {
        "name": "lora_bias",
        "value": "none"
    },
    {
        "name": "load_in_4bit",
        "value": "false"
    },
    {
        "name": "bnb_4bit_compute_dtype",
        "value": "bfloat16"
    },
    {
        "name": "bnb_4bit_quant_type",
        "value": "fp4"
    },
    {
        "name": "bnb_4bit_use_double_quant",
        "value": "false"
    },
    {
        "name": "trust_remote_code",
        "value": "false"
    },
    {
        "name": "use_auth_token",
        "value": "true"
    },
    {
        "name": "trainer_ram",
        "value": "100"
    },
    {
        "name": "trainer_gpus",
        "value": "1"
    },
    {
        "name": "trainer_cores",
        "value": "8"
    },
    {
        "name": "ephemeral_storage",
        "value": "99"
    },
    {
        "name": "finetuner_image",
        "value": "aimle2e/fine-tuner"
    },
    {
        "name": "finetuner_tag",
        "value": "gemma_7b_inst_v5"
    },
    {
        "name": "finetuner_image_policy",
        "value": "Always"
    },
    {
        "name": "sku_type",
        "value": ""
    },
    {
        "name": "wandb_key",
        "value": ""
    },
    {
        "name": "wandb_project",
        "value": ""
    },
    {
        "name": "wandb_run_name",
        "value": ""
    },
    {
        "name": "learning_rate",
        "value": "0.00002"
    },
    {
        "name": "max_train_samples",
        "value": "10"
    },
    {
        "name": "max_eval_samples",
        "value": "10"
    },
    {
        "name": "workspace",
        "value": "/mnt/workspace"
    },
    {
        "name": "torch_cache",
        "value": "/mnt/workspace/torch"
    },
    {
        "name": "transformers_cache",
        "value": "/mnt/workspace/hf"
    },
    {
        "name": "output_dir",
        "value": "/mnt/workspace/finetunes/"
    },
    {
        "name": "prompt_template_base64",
        "value": ""
    },
    {
        "name": "resume",
        "value": "true"
    }
],
"run_type": "scheduled_job",
"periodic_schedule": {
    "interval_second": 86400,
    "start_time": "2024-07-08T18:25:42.652652Z",
    "end_time": "2024-07-09T18:25:42.653653Z"
}
}

Disable Schedule Run

To disable the Schedule Run, send a PUT request to the Fine Tuning Scheduled Run Endpoint

https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/jobs/{{job_id}}/?action=disable&apikey={{tapi_key}}

PythonCurl

import requests

url = "https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/jobs/{{job_id}}/?action=disable&apikey={{tapi_key}}"

payload={}
headers = {
'Authorization': 'Bearer {{Token}}',
'Content-Type': 'application/json',
}

response = requests.request("PUT", url, headers=headers, data=payload)

print(response.text)

Request HeadersResponse Headers

Content-Type: application/json
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAi...

Delete Scheduled Run

To Delete the Scheduled Run, send a DELETE request to the Fine Tuning Scheduled Run Endpoint

https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/jobs/{{job_id}}/?apikey={{tapi_key}}

PythonCurl

import requests

url = "https://api.e2enetworks.com/myaccount/api/v1/gpu/projects/{{project_id}}/pipelines/jobs/{{job_id}}/?apikey={{tapi_key}}"

payload={}
headers = {
'Authorization': 'Bearer {{Token}}',
'Content-Type': 'application/json',
}

response = requests.request("DELETE", url, headers=headers, data=payload)

print(response.text)

Request HeadersResponse Headers

Content-Type: application/json
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAi...