Skip to main content

Generative AI API

TIR's Generative AI (GenAI) API offers a suite of ready-to-use inference endpoints, enabling seamless interaction with advanced models for text-to-text, speech-to-text, embeddings, and text-to-speech tasks. These APIs provide developers with the tools to easily integrate generative AI capabilities into their applications.

Landing Page

Key Features​

  • Cost Effective: Pay only for the requests you make, helping you manage costs while scaling as needed.
  • Always Ready: GenAI APIs are available anytime, with no long-term commitments or setup overhead. Integrate instantly without lock-in.
  • Playground: Explore and test model capabilities directly in TIR's UI. Just pick a model and start experimenting.
info

Billing parameters may vary depending on the model. For detailed pricing, select a model and check the usage section.

Playground​

You can interact with our chat models directly via the Playground. Select a model, configure the parameters, and start generating responses.

Model Listing

After selecting a model, you’ll see the home screen.
In the bottom-right corner, you’ll find Advanced Parameters that allow you to fine-tune the model’s behavior:

Advanced Parameters​

  • Top P (Nucleus Sampling): Controls diversity by limiting the sampling pool to the top probability mass p. Lower values make the model more focused; higher values increase randomness.
  • Temperature: Adjusts the randomness of responses. Lower values (e.g., 0.2) make outputs more deterministic, while higher values (e.g., 0.8) make them more creative.
  • Max Tokens: The maximum number of tokens the model can generate in a single response. Higher values allow longer outputs.
  • Presence Penalty: Increases the likelihood of introducing new topics. Higher values discourage repetition.
  • Frequency Penalty: Reduces the model’s tendency to repeat the same words or phrases. Higher values encourage variety.

Model page

Bulk Prompts​

GenAI supports bulk prompt uploads, enabling you to process multiple prompts in parallel.
To use this feature, click on the Click here option as shown below. All bulk prompt activities can be monitored in Async History.

Bulk prompt

Uploading Dataset​

1. Select a dataset.

DataSet Selection

2. Upload your CSV file within that dataset.

CSV File Requirements
  • Must contain a single column with the header prompts.
  • Each row under this column should contain one prompt.
  • File must follow a standard CSV structure (sample.csv provided).
  • Maximum file size: 500 MB
  • Format supported: CSV only
  • Ensure UTF-8 encoding for special characters.

Uploading CSV File

Async History​

Once uploaded, you can track the progress in the Async History section.

Statuses include:

  • Processing: Prompts currently being processed
  • Completed: Prompts successfully processed
  • Queued: Prompts waiting in line
  • Failed: Prompts that could not be processed

Each row includes Download and Delete options, which are enabled only for completed prompts.

You can also use Clear All to remove all failed and completed prompts at once.

Async history

Integration Overview​

How to Integrate GenAI API with Your Application?​

Integration methods vary depending on the model you choose.
Detailed steps for each model can be found in the Playground or API section.

Available Integration Options​

  1. REST API: This method is compatible with all models. Use the REST API endpoints for straightforward integration.
  2. OpenAI SDK: All LLMs (text generation) models such as Llama and Mistral on TIR GenAI are OpenAI compatible. You can integrate by updating the OPENAI_BASE_URL and OPENAI_API_KEY in your application if it uses the OpenAI SDK. Start with sample code provided on TIR's UI.
  3. TIR SDK: This is a dedicated SDK for launching and managing services on E2E network's TIR platform.

We will go through all the integration options in this document. To access the model, you need to create a token.

Generating Token To Access GenAI Models​

  1. Navigate to the API Token section on TIR's UI using the side navbar.

  2. Create a new token.

  3. Copy the Auth Token and API Key.

    Copy Token


info

OPENAI_API_KEY for the OpenAI compatible model is the Auth Token copied in the previous step.

Accessing GenAI using REST API​

For this example, we will use the Llama 4 Scout 17B 16E Instruct model. Mistral is a text generation large language model.

  1. Select the Llama 4 Scout 17B 16E Instruct card in the GenAI section.

    MLlama 4 Scout 17B 16E Instruct

  2. Click on Get Code and open the HTTP tab to find the cURL request for the model.

    Get Code

    HTTP Tab

  3. Copy the cURL request into any API testing tool. For this tutorial, we'll use Postman. Before making the request, add the Auth token (generated in the Generating Token to Access GenAI Models section) to the Authorization Header.


info

For all API requests, the Auth token is of type Bearer.

Authorization in Postman

  1. After adding the token, modify the payload based on your requirements and send the request. You can also generate sample code for various languages in Postman's Code section.

    Postman Response

Accessing GenAI using OpenAI SDK​

For this example, we will use the DeepSeek V3 model. Llama is a text generation large language model.

  1. Select the DeepSeek V3 card in the GenAI section and go to Get Code tab.

    DeepSeek V3

    DeepSeek Get Code

  2. For this tutorial, we'll write the script in Python. Open the Python tab and copy the sample Python code.

    Python Tab

  3. Install the OpenAI package using pip.

 pip install -U openai

Accessing GenAI using TIR SDK

Step 1: Generate Token​

Paste the sample code in a Python file and change the value of the api_key parameter passed in the OpenAI client with the Auth Token generated in the Generating Token to Access GenAI Models section.

Python Script

Step 2: Run the Python Script​

Run the Python script, and the response will be printed.

Python Response

Using Whisper Large V3 Model​

For this example, we will use the Whisper Large V3 model. Whisper is a speech-to-text model that can be used for tasks like transcription and translation.

Step 1: Select Whisper Large V3​

Select the Whisper Large V3 card in the GenAI section.

Whisper Large V3 Card

Step 2: Copy Sample Code​

TIR SDK is written in Python. Head to the API tab to copy the sample code. Add the path of the audio file in the input field of the data dictionary and tweak other parameters.

API Tab

Python Script for Whisper

Step 3: Set Environment Variables​

Before running the script, export the environment in the terminal. Replace env values for E2E_TIR_ACCESS_TOKEN and E2E_TIR_API_KEY with the token generated in the Generating Token to Access GenAI Models section.

Set Environment Variables

Step 4: Run the Script​

Run the script, and the response will be printed.

Whisper Response

How to Check Usage & Pricing of GenAI API?​

Step 1: Select any Gen AI Model​

For example, select the Whisper Large V3 model card in the GenAI section.

Vector Embeddings Model Card

Step 2: Check Usage or Pricing​

Open the Usage tab or click on Check Pricing.

Check Pricing

How To Guides​