Our firm uses custom logic that makes a task run return a “Skipped” state to support partial flow runs while preserving task-level dependencies.
To customize the set of tasks to include, we utilize the “parameter” of flow run deployments. We define our flows like:
@flow
def some_flow(partial_run_tasks=None):
some_task()
This works great as now people can simply put a list of tasks they want to run in the UI’s Custom run tab, but it is also a bit weird to implement, as
- the argument partial_run_tasks needs to be added to all our flows
- it is not used anywhere in the flow function itself (tasks determine if they are in this list by accessing FlowRunContext.flow_run.parameters)
We feel this parameter function more like a context variable: it is set at a flow run level, shared to all task runs, and only used by the task runs. In 1.0 I remember we were able to add custom run contexts to a flow run from the UI, would a similar feature be possible in 2.0? Is there any smarter ways to make all our flows accept this context var other than adding it to all the flow function’s signature?