Step by Step Guide to Pipeline
Introduction
In the context of Artificial Intelligence (AI), a pipeline refers to a series of data processing steps or operations that are performed in sequence to achieve a specific AI task or goal. An AI pipeline typically involves several stages, each with a specific function, and it is designed to process and transform input data into meaningful output. Each stage in the pipeline plays a crucial role in the overall AI process, and the effectiveness of the pipeline depends on the quality of data, the choice of algorithms, and the expertise in designing and optimizing each step. AI pipelines are commonly used in various applications, including machine learning, natural language processing, computer vision, and more
What is Pipeline
TIR Pipelines offer a way to write scalable, serverless and asynchronous training jobs based on docker containers. The supported formats include Argo and Kubeflow Pipelines Templates.
You no longer have to worry about the reliability of training of jobs as TIR pipelines offer best-in class retry function. This allows you to restart a job without losing completed work.
Additionally, TIR pipelines also support unlimited re-runs, stored results (in EOS buckets) and all resource plans (CPU and GPU).
How to create a Pipeline
To create the Pipeline, first, the user should navigate to the sidebar section and select “Pipelines”. Upon selecting Pipelines, a dropdown menu will appear, featuring an option labeled “Pipeline”.
Upon clicking the “Pipeline” option, the user will be directed to the “Manage Pipelines” page.
Click on “GET STARTED” button.
If you click on “Get started” button, the default pipeline will be created.
Click on “CREATE PIPELINE” button.
After redirect to the “Manage Pipelines”, On this page, users can locate and click on the “CREATE PIPELINE” button to create Pipeline.
After clicking the ‘CREATE PIPELINE’ button, the ‘Create Pipeline’ page will open. On this page, you can create Pipeline with two options ‘Create a new pipeline’ or ‘Create a new pipeline version under an existing pipeline’.
Create a new pipeline
To create a new pipeline, you have to upload a .yaml file and give a description and click on “UPLOAD” button. Avoid to introduce additional nodes or commands, as such additions could lead to conflicts between nodes within their YAML file.
After successfully created pipeline, you can see the below screen with pipeline versions and pipeline details. You can also create Run and view Run.
Create Run For a selected pipeline, you can create run by clicking the plus icon and you will be redirected to “Create Run” page.
View Runs For a selected pipeline, you can view run by clicking the arrow icon and you will be redirected to “Manage Run” page. In that page you can view the created runs.
Pipeline Versions
You can see the Pipeline Versions of selecting particular pipeline under “Pipeline Versions” tab.You can create a version by clicking “CREATE VERSION” button.After successully created pipeline versions, you can see the list.
CREATE VERSIONS
To create a new pipeline version for an existing pipeline, select pipeline then click “Create Version” under pipeline Versions.
After clicking Create Version, you Create a new pipeline, you will see create pipeline screen.In that you have to upload a file type of .yaml click on upload.
You can see the pipeline versions below.
Create Run
You can create a run for a particular for a particular pipeline versions. For that you have to click on Create Run.
After clicking on “Create Run “, you will see the “Create Run” screen. In that you can fill the run details and click on finish.
View Run
After successfully created Run for a particular pipeline versions, you can view that run by clicking “View Runs”.
Pipeline Versions Actions
You can delete pipeline version by clicking delete icon.
Actions
Create Version
To create versions for a particular pipeline, select pipeline then click “Create Version” under Actions.
Delete
To delete pipeline, select pipeline then click “Delete” under Actions.
After clicking delete icon, you can see the Delete Pipeline popup then click on “DELETE” button.
Create a new pipeline version under an existing pipeline
To create a new pipeline version under an existing pipeline click on “Create a new pipeline version under an existing pipeline” then select the pipeline in which you want to upload new version and upload .yaml file and click on “UPLOAD” button.
After successfully created version, you can see the pipeline versions under “Pipeline Versions” tab.
CREATE VERSIONS
After clicking Create Version, you Create a new pipeline, you will see create pipeline screen.In that you have to upload a file type of .yaml click on upload.
You can see the pipeline versions below.
Create Run
You can create a run for a particular for a particular pipeline versions. For that you have to click on Create Run.
After clicking on “Create Run “, you will see the “Create Run” screen. In that you can fill the run details and click on finish.
View Run
After successfully created Run for a particular pipeline versions, you can view that run by clicking “View Runs”.
Pipeline Versions Actions
You can delete pipeline version by clicking delete icon.
Actions
Create Version
To create versions for a particular pipeline, select pipeline then click “Create Version” under Actions.
Delete
To delete pipeline, select pipeline then click “Delete” under Actions.
After clicking delete icon, you can see the Delete Pipeline popup then click on “DELETE” button.