Latest Workflow Orchestration topics

Topic	Replies	Views	Activity
Does Prefect support caching on a flow level rather than only on a task level? How to avoid task run recomputation of several tasks in a flow? prefect-1-0 , caching , results , checkpointing	7	322	October 21, 2022
Can I set a custom name of a task or task run at run-time in Prefect 2.0? prefect-2-0 , task-name , task-args , task_run_name	3	442	October 18, 2022
Using Prefect Client and Prefect Context for event driven workflows prefect-2-0 , context , event-driven , orchestrator-pattern , api	0	319	September 29, 2022
Does Prefect 2.0 support filters and adjustments e.g. to restrict the schedule to only run during business hours? migration-guide , prefect-2-0 , local-development , scheduling , interval-schedule , cron , unit-testing , rrule	0	332	April 29, 2022
How can I create a subflow? migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , getting-started	0	811	January 27, 2022
How to pass data between subflows? prefect-2-0 , subflows , data-dependencies	0	320	April 8, 2022
How can I attach a schedule to a flow? migration-guide , prefect-1-0 , prefect-2-0 , deployment , parameters , scheduling , getting-started , interval-schedule	4	535	August 19, 2022
Is it possible to skip a new scheduled flow run if a previous flow run of the same flow is still in progress? prefect-1-0 , state_handlers , skipping-task-or-flow-runs	5	488	August 10, 2022
How to override default parameter values when creating a flow run from deployment? prefect-2-0 , deployment , parameters , create_flow_run	5	227	August 8, 2022
How can I organize my flows based on my business logic or team structure? organizational , migration-guide , prefect-1-0 , prefect-2-0 , cloud-1-0 , deployment , tags , projects	0	835	January 25, 2022
How to take action on a state change of a task run? (task-level state handler) migration-guide , prefect-1-0 , prefect-2-0 , automations , state_handlers , states , slack-notifications , notifications , failure , getting-started , email-notifications	11	1398	August 1, 2022
How to map over flows with various parameter values? prefect-1-0 , parameters , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , mapping	1	829	July 25, 2022
What is the default TaskRunner (executor)? migration-guide , prefect-1-0 , prefect-2-0 , executor , task-runner , defaults	1	743	July 21, 2022
What's the role of agents and work queues, and how the concept of agents differ between Prefect 1.0 and 2.0? migration-guide , prefect-1-0 , prefect-2-0 , agent , work-queue , work-queue-agent-relationship	0	680	April 5, 2022
How to securely store secrets in Prefect 2.0? prefect-2-0 , secrets , blocks	0	637	July 20, 2022
How to add retries when processing files and know which files failed to get processed? prefect-2-0 , retries , state-dependencies , final-state , fie-processing	0	176	July 13, 2022
Create a Customized Decorator for Task prefect-2-0	0	127	July 11, 2022
How can I parallelize execution across 8 CPU cores? migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , local-dask-executor , infrastructure , multithreading , multiprocessing	0	483	January 31, 2022
How can I run tasks in parallel? migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , infrastructure , multithreading , multiprocessing , basics	0	587	January 31, 2022
Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level? migration-guide , prefect-2-0 , dag-flow-structure , subflows , getting-started , task-decorator , best-practices , flow-decorator	0	369	March 17, 2022
Can one add context in prefect2 during the flow? prefect-2-0	1	137	June 27, 2022
Use GraphQL to Change State of Flow Run prefect-1-0 , graphql	0	191	June 16, 2022
Singleton task dependence	5	439	June 3, 2022
Upon Flow failure, use an Automation to start another Flow prefect-1-0 , automations	0	139	May 24, 2022
How to configure retries on a task from a Prefect Collection (in Prefect 2.0)? prefect-2-0 , shell , retries , prefect-collections , with_options , prefect_shell	0	197	May 22, 2022
Cloud - agent security prefect-1-0	1	143	May 18, 2022
How to ensure that exceptions and errors are logged with ERROR log level rather than INFO? prefect-1-0 , logging , signals , exception-handling , log-level	0	178	May 16, 2022
What are best practices to build unit tests and integration tests for Prefect flows and for data pipelines in general? prefect-1-0 , prefect-2-0 , best-practices , data-quality , data-validation , unit-testing	0	788	May 10, 2022
How can I test flows and tasks in Prefect 2.0? prefect-2-0 , unit-testing	2	359	May 5, 2022
Why results need to be configured in order to use retries? prefect-1-0 , retries , results	0	245	May 3, 2022

Does Prefect support caching on a flow level rather than only on a task level? How to avoid task run recomputation of several tasks in a flow?

7

322

October 21, 2022

Can I set a custom name of a task or task run at run-time in Prefect 2.0?

prefect-2-0 , task-name , task-args , task_run_name

3

442

October 18, 2022

Using Prefect Client and Prefect Context for event driven workflows

prefect-2-0 , context , event-driven , orchestrator-pattern , api

0

319

September 29, 2022

Does Prefect 2.0 support filters and adjustments e.g. to restrict the schedule to only run during business hours?

migration-guide , prefect-2-0 , local-development , scheduling , interval-schedule , cron , unit-testing , rrule

0

332

April 29, 2022

How can I create a subflow?

migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , getting-started

0

811

January 27, 2022

How to pass data between subflows?

prefect-2-0 , subflows , data-dependencies

0

320

April 8, 2022

How can I attach a schedule to a flow?

migration-guide , prefect-1-0 , prefect-2-0 , deployment , parameters , scheduling , getting-started , interval-schedule

4

535

August 19, 2022

Is it possible to skip a new scheduled flow run if a previous flow run of the same flow is still in progress?

prefect-1-0 , state_handlers , skipping-task-or-flow-runs

5

488

August 10, 2022

How to override default parameter values when creating a flow run from deployment?

prefect-2-0 , deployment , parameters , create_flow_run

5

227

August 8, 2022

How can I organize my flows based on my business logic or team structure?

organizational , migration-guide , prefect-1-0 , prefect-2-0 , cloud-1-0 , deployment , tags , projects

0

835

January 25, 2022

How to take action on a state change of a task run? (task-level state handler)

migration-guide , prefect-1-0 , prefect-2-0 , automations , state_handlers , states , slack-notifications , notifications , failure , getting-started , email-notifications

11

1398

August 1, 2022

How to map over flows with various parameter values?

prefect-1-0 , parameters , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , mapping

1

829

July 25, 2022

What is the default TaskRunner (executor)?

migration-guide , prefect-1-0 , prefect-2-0 , executor , task-runner , defaults

1

743

July 21, 2022

What's the role of agents and work queues, and how the concept of agents differ between Prefect 1.0 and 2.0?

migration-guide , prefect-1-0 , prefect-2-0 , agent , work-queue , work-queue-agent-relationship

0

680

April 5, 2022

How to securely store secrets in Prefect 2.0?

prefect-2-0 , secrets , blocks

0

637

July 20, 2022

How to add retries when processing files and know which files failed to get processed?

prefect-2-0 , retries , state-dependencies , final-state , fie-processing

0

176

July 13, 2022

Create a Customized Decorator for Task

prefect-2-0

0

127

July 11, 2022

How can I parallelize execution across 8 CPU cores?

migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , local-dask-executor , infrastructure , multithreading , multiprocessing

0

483

January 31, 2022

How can I run tasks in parallel?

migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , infrastructure , multithreading , multiprocessing , basics

0

587

January 31, 2022

Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level?

migration-guide , prefect-2-0 , dag-flow-structure , subflows , getting-started , task-decorator , best-practices , flow-decorator

0

369

March 17, 2022

Can one add context in prefect2 during the flow?

prefect-2-0

1

137

June 27, 2022

Use GraphQL to Change State of Flow Run

prefect-1-0 , graphql

0

191

June 16, 2022

Singleton task dependence

5

439

June 3, 2022

Upon Flow failure, use an Automation to start another Flow

prefect-1-0 , automations

0

139

May 24, 2022

How to configure retries on a task from a Prefect Collection (in Prefect 2.0)?

prefect-2-0 , shell , retries , prefect-collections , with_options , prefect_shell

0

197

May 22, 2022

Cloud - agent security

prefect-1-0

1

143

May 18, 2022

How to ensure that exceptions and errors are logged with ERROR log level rather than INFO?

prefect-1-0 , logging , signals , exception-handling , log-level

0

178

May 16, 2022

What are best practices to build unit tests and integration tests for Prefect flows and for data pipelines in general?

prefect-1-0 , prefect-2-0 , best-practices , data-quality , data-validation , unit-testing

0

788

May 10, 2022

How can I test flows and tasks in Prefect 2.0?

prefect-2-0 , unit-testing

2

359

May 5, 2022

Why results need to be configured in order to use retries?

prefect-1-0 , retries , results

0

245

May 3, 2022

Workflow Orchestration

Don't Panic

Read our Privacy Policy