# Flow
# Flow
class
prefect.core.flow.Flow
(name, schedule=None, environment=None, storage=None, tasks=None, edges=None, reference_tasks=None, state_handlers=None, on_failure=None, validate=None, result_handler=None, result=None)[source]The Flow class is used as the representation of a collection of dependent Tasks. Flows track Task dependencies, parameters and provide the main API for constructing and managing workflows.
Initializing Flow example:
class MyTask(Task):
def run(self):
return "hello"
task_1 = MyTask()
flow = Flow(name="my_flow", tasks=[task_1])
flow.run()
Initializing Flow as context manager example:
@task
def my_task():
return "hello"
with Flow("my_flow") as flow:
task_1 = my_task()
flow.run()
Args:
name (str)
: The name of the flow. Cannot beNone
or an empty stringschedule (prefect.schedules.Schedule, optional)
: A default schedule for the flowenvironment (prefect.environments.Environment, optional)
: The environment that the flow should be run in. IfNone
, aRemoteEnvironment
will be created.storage (prefect.environments.storage.Storage, optional)
: The unit of storage that the flow will be written into.tasks ([Task], optional)
: If provided, a list of tasks that will initialize the flowedges ([Edge], optional)
: A list of edges between tasksreference_tasks ([Task], optional)
: A list of tasks that determine the final state of a flowresult (Result, optional, RESERVED FOR FUTURE USE)
: the result instance used to retrieve and store task results during executionresult_handler (ResultHandler, optional, DEPRECATED)
: the handler to use for retrieving and storing state results during executionstate_handlers (Iterable[Callable], optional)
: A list of state change handlers that will be called whenever the flow changes state, providing an opportunity to inspect or modify the new state. The handler will be passed the flow instance, the old (prior) state, and the new (current) state, with the following signature:state_handler(flow: Flow, old_state: State, new_state: State) -> Optional[State]
If multiple functions are passed, then thenew_state
argument will be the result of the previous handler.on_failure (Callable, optional)
: A function with signaturefn(flow: Flow, state: State) -> None
which will be called anytime this Flow enters a failure statevalidate (bool, optional)
: Whether or not to check the validity of the flow (e.g., presence of cycles and illegal keys) after adding the edges passed in theedges
argument. Defaults to the value ofeager_edge_validation
in your prefect configuration file.
methods: |
---|
prefect.core.flow.Flow.add_edge (upstream_task, downstream_task, key=None, mapped=False, validate=None)[source] |
Add an edge in the flow between two tasks. All edges are directed beginning with an upstream task and ending with a downstream task.
|
prefect.core.flow.Flow.add_task (task)[source] |
Add a task to the flow if the task does not already exist. The tasks are uniquely identified by their
|
prefect.core.flow.Flow.all_downstream_edges ()[source] |
Returns a dictionary relating each task in the Flow to the set of all downstream edges for the task
|
prefect.core.flow.Flow.all_upstream_edges ()[source] |
Returns a dictionary relating each task in the Flow to the set of all upstream edges for the task
|
prefect.core.flow.Flow.chain (*tasks, validate=None)[source] |
Adds a sequence of dependent tasks to the flow; each task should be provided as an argument (or splatted from a list).
|
prefect.core.flow.Flow.copy ()[source] |
Create and returns a copy of the current Flow. |
prefect.core.flow.Flow.diagnostics (include_secret_names=False)[source] |
Get flow and Prefect diagnostic information
|
prefect.core.flow.Flow.downstream_tasks (task)[source] |
Get all of the tasks downstream of a task
|
prefect.core.flow.Flow.edges_from (task)[source] |
Get all of the edges leading from a task (i.e., the downstream edges)
|
prefect.core.flow.Flow.edges_to (task)[source] |
Get all of the edges leading to a task (i.e., the upstream edges)
|
prefect.core.flow.Flow.get_tasks (name=None, slug=None, tags=None, task_type=None)[source] |
Helper method for retrieving tasks from this flow based on certain attributes. The intersection of all provided attributes is taken, i.e., only those tasks which match all provided conditions are returned.
|
prefect.core.flow.Flow.load (fpath)[source] |
Reads a Flow from a file that was created with
|
prefect.core.flow.Flow.parameters ()[source] |
Returns any parameters of the flow.
|
prefect.core.flow.Flow.reference_tasks ()[source] |
A flow's "reference tasks" are used to determine its state when it runs. If all the reference tasks are successful, then the flow run is considered successful. However, if any of the reference tasks fail, the flow is considered to fail. (Note that skips are counted as successes; see the state documentation for a full description of what is considered failure, success, etc.)
|
prefect.core.flow.Flow.register (project_name=None, build=True, labels=None, set_schedule_active=True, version_group_id=None, no_url=False, **kwargs)[source] |
Register the flow with Prefect Cloud; if no storage is present on the Flow, the default value from your config will be used and initialized with
|
prefect.core.flow.Flow.replace (old, new, validate=True)[source] |
Performs an inplace replacement of the old task with the provided new task.
|
prefect.core.flow.Flow.root_tasks ()[source] |
Get the tasks in the flow that have no upstream dependencies; these are the tasks that, by default, flow execution begins with.
|
prefect.core.flow.Flow.run (parameters=None, run_on_schedule=None, runner_cls=None, **kwargs)[source] |
Run the flow on its schedule using an instance of a FlowRunner. If the Flow has no schedule, a single stateful run will occur (including retries).
|
prefect.core.flow.Flow.run_agent (token=None, show_flow_logs=False, log_to_cloud=True)[source] |
Runs a Cloud agent for this Flow in-process.
|
prefect.core.flow.Flow.save (fpath=None)[source] |
Saves the Flow to a file by serializing it with cloudpickle. This method is recommended if you wish to separate out the building of your Flow from its registration.
|
prefect.core.flow.Flow.serialize (build=False)[source] |
Creates a serialized representation of the flow.
|
prefect.core.flow.Flow.set_dependencies (task, upstream_tasks=None, downstream_tasks=None, keyword_tasks=None, mapped=False, validate=None)[source] |
Convenience function for adding task dependencies.
|
prefect.core.flow.Flow.set_reference_tasks (tasks)[source] |
Sets the
|
prefect.core.flow.Flow.sorted_tasks (root_tasks=None)[source] |
Get the tasks in this flow in a sorted manner. This allows us to find if any cycles exist in this flow's DAG.
|
prefect.core.flow.Flow.terminal_tasks ()[source] |
Get the tasks in the flow that have no downstream dependencies
|
prefect.core.flow.Flow.update (flow, validate=None)[source] |
Take all tasks and edges in another flow and add it to this flow
|
prefect.core.flow.Flow.upstream_tasks (task)[source] |
Get all of the tasks upstream of a task
|
prefect.core.flow.Flow.validate ()[source] |
Checks that the flow is valid.
|
prefect.core.flow.Flow.visualize (flow_state=None, filename=None, format=None)[source] |
Creates graphviz object for representing the current flow; this graphviz object will be rendered inline if called from an IPython notebook, otherwise it will be rendered in a new window. If a
|
This documentation was auto-generated from commit n/a
on June 17, 2020 at 17:27 UTC