Get started with Pipelines

Turn a YAML file into a pipeline, start a run of it on a plan you choose, and read the run’s logs.

Before you begin

You need a TIR project you can create pipelines in.

Write a pipeline file

A pipeline file is a .yaml file in Argo Workflows format, or one compiled from the Kubeflow Pipelines SDK. To start, save this Hello World workflow as hello-world.yaml. It is the same example the See Example panel shows on the create page, and its one step prints a greeting:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
spec:
  entrypoint: whalesay
  templates:
    - name: whalesay
      container:
        image: docker/whalesay
        command: [cowsay]
        args: ["hello world"]

entrypoint names the step that runs first, and each template runs one container image with its command and arguments. To run your own image instead, see Run a Docker image as a pipeline.

Create the pipeline

1. In TIR, open your project. In the side menu, under PRODUCTS, select Pipeline.
2. Select Create Pipeline. For a project’s first pipeline, this button is on the first page you see.
3. Leave Create a new pipeline selected. In Pipeline Name, keep the generated name or type your own, up to 30 characters.
4. (Optional) In Description, add up to 300 characters.
5. In Upload Pipeline File, add hello-world.yaml.
6. Select Create Pipeline.

You return to Manage Pipelines, and the pipeline is in the list with the Pipeline Type Argo.

Create a run

1. In the pipeline’s row, select the icon in the Create Run column.
2. In Run Name, keep the generated name or type your own, up to 30 characters.
3. In Experiment, choose an experiment. If the list is empty, use the link beside Experiment to create one. An experiment is a name you group runs under.
4. Under Select Computing Resource, choose the GPU or CPU tab, pick a plan, and pick a price under Instance Pricing. Choose a CPU plan for steps that move or prepare data and a GPU plan for training.
5. Select Create Run. The Hello World file defines no parameters. A file that defines parameters lists them under Run Parameters, where you can change their values.

See the result

Back on Manage Pipelines, select the pipeline, then the Pipeline Runs tab. Select the run’s name to open Run Details, select the whalesay step in Graph, and open Logs. The page keeps updating until the run finishes, and the log shows the greeting. The run is billed by the minute while its step runs, and billing stops when the step ends.

To follow a run more closely, watch it and read its logs. To start the same run every hour or every day, schedule a run.