Generative AI API

TIR’s Generative AI (GenAI) API offers a suite of ready-to-use inference endpoints, enabling seamless interaction with advanced models for text-to-text, speech-to-text, embeddings, and text-to-speech tasks. These APIs provide developers with the tools to easily integrate generative AI capabilities into their applications.

../../_images/landing_page1.png

Key features:

  • Cost Effectiveness: You’re only charged for the requests you make. Allowing you to control expenses while scaling according to your needs.

  • Ready to Use: GenAI APIs are always available and can be accessed anytime, providing instant integration into your applications with no commitments or long-term contracts. Use them as needed without worrying about setup or lock-in periods.

  • Playground : Test and explore models capabilities directly on TIRs UI in the Playground section after selecting the Model

Note

Billing parameters may vary based on the model you use. For more information on pricing, select the model you are interested in and refer to the usage section.

Quick Start

How to Integrate GenAI API with your application ?

Integration methods vary depending on the model you use. Detailed steps for each model can be found in the Playground or API section.

Here are the available integration options:

  1. Rest API: This method is compatible with ll models. Use the REST API endpoints for straightforward integration.

  2. OpenAI SDK: All LLMs (text generation) models such as Llama and Mistral on TIR GenAI are OpenAI compatible. you can integrate by updating the OPENAI_BASE_URL and OPENAI_API_KEY in your application if it uses openai sdk. Start with sample code provided on TIRs UI.

  3. TIR SDK: It is a dedicated SDK for lauching and manging services on E2E network’s TIR platform

We will go through all the integrations options in this doc. To access the model you need to create Token.

Generating Token To Access GenAI Models

  1. Navigate to the API Token Section on TIRs UI using the side navbar
    ../../_images/landing_page2.png

  2. Create new Token

  3. Copy the Auth Token and API Key.
    ../../_images/copy_token.png

Note

OPENAI_API_KEY for the openai compatible model is the auth token copied in the previous step

Accessing GenAI using Rest API

For this example we will use Mistral 7B Instruct v0.3 Model. Mistral is Text generations Large language model

  1. Select Mistral 7B Instruct v0.3 card in the GenAI section
    ../../_images/lp.png
  2. Open the HTTP tab to find the cURL request for the model
    ../../_images/http_tab.png

  3. Copy the cURL request into any API testing tool. For this tutorial, we’ll use Postman. Before making the request, add the Auth token (generated in the Generating Token to Access GenAI Models section) to the Authorization Header.

    Note

    For all API requests, the Auth token is of type Bearer.

    ../../_images/auth_postman.png

  4. After adding the token, modify the payload based on your requirement send the request. You can also generate sample code for various languages in Postman’s Code section.

    ../../_images/response_postman.png

Accessing GenAI using OpenAI SDK

For this example we will use Llama 3.1 8B Instruct Model. Llama is Text generations Large language model. #. Select Llama 3.1 8B Instruct card in the GenAI section

../../_images/lp1.png
  1. For this tutorial, we’ll write the script in Python. Open the Python tab and copy the sample Python code
    ../../_images/python_tab.png

  2. Install OpenAI package using pip

    pip install -U openai

  3. Paste the sample code in python file and change value the api_key parameter passed in openai client with the Auth Token generated in the Generating Token to Access GenAI Models section

    ../../_images/python_script.png

  4. Run the python script and response will be printed

    ../../_images/python_response.png

Accessing GenAI using TIR SDK

For this example we will use Whisper Large V3 Model. Whisper is Speech to text model which can used for tasks like Transcribe, Translate. #. Select Whisper Large V3 card in the GenAI section

../../_images/lp2.png
  1. TIR SDK is written in python, so head to API tab can copy sample code. Add path of audio file in the input field of data dict and tweak other parameters
    ../../_images/api_tab.png ../../_images/python_script1.png

  2. Before running the script export environment in terminal, replace env values for E2E_TIR_ACCESS_TOKEN and E2E_TIR_API_KEY with token generated in in the Generating Token to Access GenAI Models section

    ../../_images/envs.png
  3. Run the script and the response will be printed
    ../../_images/response.png

How to Check Usage & Pricing of GenAI API ?

  1. For Example select Vector Embeddings model card in the GenAI section

    ../../_images/lp3.png
  2. Open Usage tab or click on Check Pricing
    ../../_images/play.png
  3. Open Usage tab or click on Check Pricing. You can also specify custom interval to check billing between specific dates.
    ../../_images/usage_custom.png

Note

When Multiple Billing parameter are associated with model it will be mentioned in the usage table along with their costing