E2E CloudPipelines FAQ
Pipeline files and versions
Section titled “Pipeline files and versions”Which file formats does a pipeline accept?
Section titled “Which file formats does a pipeline accept?”A .yaml file in Argo Workflows format, or one compiled from the Kubeflow Pipelines SDK. A file
that contains containerSet is refused. See Create a
pipeline.
Does TIR keep the workflow name in my file?
Section titled “Does TIR keep the workflow name in my file?”No. TIR names the workflow itself, in place of the name or generateName in your file.
How do I change a pipeline without losing the old one?
Section titled “How do I change a pipeline without losing the old one?”Create a new version. A run names the version it uses, so earlier versions stay available. See Create a pipeline version.
Can I run my own Docker image?
Section titled “Can I run my own Docker image?”Yes. Name the image, command, and arguments in an Argo Workflow file; a private image also needs its pull secret from Container Registry. See Run a Docker image.
Can a pipeline move data between object storage and a PFS?
Section titled “Can a pipeline move data between object storage and a PFS?”Yes, with two ready-made files. See Transfer data between EOS and PFS.
Runs and plans
Section titled “Runs and plans”Should I choose a CPU or a GPU plan?
Section titled “Should I choose a CPU or a GPU plan?”Use a CPU plan for steps that prepare or move data and a GPU plan for training. See Create a run.
Can I choose a different plan for each step?
Section titled “Can I choose a different plan for each step?”No. A run has one CPU or GPU plan, and every step is billed at that plan’s price. To use different plans, split the work into separate runs. See Billing and plans for pipeline runs.
What does Retry do, and when is it available?
Section titled “What does Retry do, and when is it available?”Retry is available while a run’s status is Failed. It starts that run again and skips the steps that finished. Terminate is available while the status is Running. See Retry, terminate, or delete a run.
Where do I find a run’s logs?
Section titled “Where do I find a run’s logs?”Open the run, select a step in Graph, and select Logs. See Watch a run and read its logs.
Where are a run’s artifacts stored?
Section titled “Where are a run’s artifacts stored?”In the object storage bucket TIR sets up for your project when you create your first pipeline. Data you keep in object storage is billed separately. See How pipelines, versions, runs, and experiments fit together.
What happens to runs when I delete a version or a pipeline?
Section titled “What happens to runs when I delete a version or a pipeline?”They are deleted too. Deleting a version deletes its runs, and deleting a pipeline deletes all its versions and runs. See Delete a pipeline or a version.
Schedules
Section titled “Schedules”How often can a schedule repeat?
Section titled “How often can a schedule repeat?”Every hour, from 1 to 24 hours, or every day. See Schedule a run.
How long does a schedule keep running?
Section titled “How long does a schedule keep running?”Until its end time. Even with Never chosen, the create-run form sets an end time of 23:55 on the day after you open the form. Choose On specific date and set the End Time you want. See Schedule a run.
Billing
Section titled “Billing”How is a run billed?
Section titled “How is a run billed?”By the minute at the plan’s hourly price, for the time each step runs. A step’s billing begins when its container starts. See Billing and plans for pipeline runs and E2E’s pricing page.
Do I pay when nothing is running?
Section titled “Do I pay when nothing is running?”Pipelines, versions, experiments, and schedules themselves are free. You pay for a schedule only through the runs it starts. Data you keep in object storage is billed separately.