Skip to main content
Flow definitions are defined using Flyte’s Python library.

Flow files

Workflow basics

Flows are defined inside a Python file. In this document, we call the file workflow.py, but the name can be anything.
workflow.py
We’re using the @workflow decorator to declare a function as a flow. The method name, training_workflow, is implicitly used as the name of the flow.

Inputs & outputs

Your flow can be parameterized so that you can re-run it with new inputs without modifying the code of your flow. Inputs can be either structured data or files.
Declare file inputs and outputs using the generic FlyteFile type.To enable file rendering in the UI, this type must either:
  • Be combined with a TypeVar that defines the file extension type, for example, FlyteFile[TypeVar("csv")] and FlyteFile[TypeVar("pdf")].
  • Be a Flow Artifact like Artifact(name="My Data", type=DATA).File(name="data.csv")
Use base Python types for passing structured data through flows:str, bool, int, float, list, dict, datetime
A number of common data science types are supported as well:np.ndarray, pandas.DataFrame, pyspark.DataFrame, torch.Tensor / torch.nn.Module, sklearn.base.BaseEstimator, tf.keras.Model
Explicit, strongly typed inputs and outputs ensure that tasks pass data to each other correctly. For instance, a data processing task may emit a Pandas dataframe as output, while a training task accepts a Pandas dataframe as input. Flows analyze workflow submissions to prevent user error before execution, providing actionable feedback for incorrectly specified parameters that don’t satisfy task contracts.
Input and output types are also used for caching, data lineage tracking, and previewing in Domino.
Here we parameterize our flow by accepting a parameter (n) to our @workflow function and returning one of our results (the "results" output in sim):
workflow.py

Flow tasks

Declare tasks

Tasks are the core building blocks within a flow and are isolated within their own container during an execution. A task maps to a single Domino Job.
While all tasks trigger a unique Domino Job, there are some differences between jobs launched by a flow and standalone jobs. More specifically, for jobs launched by a flow:
  • Only Domino Dataset snapshots and NetApp Volume snapshots can be mounted, unlike standalone Domino Jobs which support snapshot mounts and read-write mounts. For data to be used in a flow, it must be part of a versioned Dataset snapshot (version 0 of a dataset, that is, the read-write directory is NOT considered a snapshot), or part of a NetApp Volume snapshot.
  • Only one snapshot of a Dataset or NetApp Volume may be mounted at a time. When leveraging the use_latest flag, the latest snapshot is mounted.
  • Snapshots are read-only and cannot be modified during a job. Any processed data that needs to be persisted should be defined and written as a task output and therefore written to the Flow blob storage.
  • Snapshots are mounted to a standard location that doesn’t include a snapshot ID in the path (the same location used for the latest version of a dataset or NetApp Volume in a workspace). For DFS projects, the path is:
    • /domino/datasets/local/{name} for local dataset snapshots.
    • /domino/datasets/{name} for shared dataset snapshots.
    • /domino/netapp-volumes/{name} for NetApp Volume snapshots.
    For Git-based projects, the path is:
    • /mnt/data/{name} for local dataset snapshots.
    • /mnt/imported/data/{name} for shared dataset snapshots.
    • /mnt/netapp-volumes/{name} for NetApp Volume snapshots.
  • Snapshots of the project code and artifacts (results) are not taken at the end of the job. Any results that need to be persisted should be defined and written as a task output and therefore written to the Flow blob storage.
  • There are two additional directories: /workflow/inputs and /workflow/outputs. These are where the task inputs/outputs of the flow are stored. A job status is considered failed if the expected task outputs are not produced by the end of execution. See the Write task code section for more details on how to write your code accordingly.
  • Stopping a job orchestrated by a Flow in the Jobs UI stops the entire flow, including other jobs that are running as part of it.
  • Completed jobs cannot be re-run through the Domino Jobs UI. They must be relaunched by re-running the task from the Flows UI.
  • Additional job metadata is captured and displayed in the Job Details to reference the flow and task that launched it.
These differences help to guarantee the reproducibility of flow executions by ensuring the triggered jobs adhere to a strict contract and remain side-effect free.
Tasks for a flow can be defined in one of the following ways:
  • Use the base DominoJobConfig and DominoJobTask classes. These provide more flexibility and direct control of the exact parameters to use.
  • Use the run_domino_job_task helper method. This offers a more user-friendly abstraction that enables definition and execution of a task in the same line of code.
In both cases, tasks trigger a Domino Job with the specified settings and return the results.

Pass data

It’s common for one task to depend on another task; that is, one task accepts an input that is produced by another task as an output. This ensures that the dependent task does not start execution until outputs from the other task are produced first. To create dependent tasks, you can use either the base classes or helper methods to define them. In the example below, note how the second task uses the output from the first task by calling data_prep_results["processed_data"].

Return outputs

You can set the output of the flow by returning it in the method. Note that defining an overall Flow output is not required and does not elevate this particular output in the UI. Please see Define Flow Artifacts for elevating important task outputs.

Write task code

Domino code execution aims for consistency across all execution modalities. To support this, flows can leverage the same data mount paths used in a workspace or standalone job. As an input, declared inputs that reside under /mnt are populated for reading. Data written back to data paths can optionally be used to create a new snapshot of the data at the end of the flow run. However, for explicit inputs and outputs, flows additionally mounts data for input and output in two special directories under /workflow. Files can be read from /workflow/inputs and written to /workflow/outputs. In addition to files, flows also allows for the passing of structured data such as dataframes and lists. These objects also show up in the /workflow directories, but require special handling to read them as structured data in Python.

Read inputs

For each input that is defined for a task, a file is created and accessible within a Job within /workflow/inputs/ in a file whose name exactly matches that of the named input. For file type inputs, the file can be read directly at the given path.
For Python non-file types (str, bool, int, list, dict, etc.), the blob contents contain the input value. Example usage:

Write outputs

Outputs defined for a task must be written into /workflow/outputs/ to a file that exactly matches the name of the output.
If the defined outputs do not exist at the end of a Domino Job the task fails.
Flow jobs do not make commits or dataset snapshots. Results should always be written as outputs. Snapshots are taken at the end of flow execution if configured.

Domino Flyte plugin

Domino provides a set of Flyte-compatible classes, as well as utility functions for defining tasks using these classes, in a way that integrates nicely with Domino Jobs. Notably, the flytekitplugins.domino library provides groups of utilities in the helpers, task, and artifact submodules.

Class constructors

The following example defines a task using the base classes.
To pin a specific branch, commit, or tag for one or more of the project’s imported Git repos on a per-task basis, pass ImportedGitRepoOverrides to DominoJobConfig (Git-based projects only):

DominoJobConfig()

The DominoJobConfig defines the configuration for the Domino Job that is triggered by the task.

DominoJobTask()

Calling the Domino Job task with the relevant inputs (data_prep_job(data_path=data_path)) runs the Domino Job and returns the results as a Promise, which can be used as an input to downstream tasks.

EnvironmentRevisionSpec()

ComputeClusterProperties()

Helpers

run_domino_job_task()

Helper methods reduce the amount of code necessary to invoke a task. Instead of separately defining a DominoJobConfig and passing it to a DominoJobTask in the examples above, use run_domino_job_task to define the task config, job, inputs and outputs all using a single function call.
To pin imported Git repos for a task, use the base DominoJobConfig and DominoJobTask classes with ImportedGitRepoOverrides. The run_domino_job_task helper also accepts imported_git_repo_overrides.

Input()

Specify a Flows Task input. Inputs include values and files, and must be declared for each Task.

Output()

Specify a Flows Task output.

Next steps

Once you have properly defined the flow, learn how to:
Last modified on August 21, 2026