Workflow Orchestration


Topic Replies Views Activity
About the Workflow Orchestration category
0 172 January 21, 2022
Edgecase Error: "Prefect Task run TASK_RUN_ID already finished"
prefect-2-0 ,  parallel-processing ,  ray ,  ray-task-runner
10 110 December 1, 2022
How can I map over a list of inputs?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner ,  mapping ,  local-dask-executor ,  getting-started ,  to-review
13 359 November 30, 2022
Prefect scheduling not working with docker-compose
prefect-2-0 ,  scheduling ,  docker-compose
5 53 November 28, 2022
Enforcing flow timeout
prefect-2-0 ,  timeout
3 39 November 23, 2022
Asynchronous parallel tasks
prefect-2-0 ,  dask ,  async
11 65 November 22, 2022
How can I run multiple subflows (or child flows) in parallel?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  parallel-processing ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  async ,  mapping ,  local-dask-executor
10 1108 November 22, 2022
How can I impose concurrency limits for flow and task runs?
organizational ,  migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  cloud-1-0 ,  concurrency-limits ,  to-review
0 524 January 27, 2022
Running subflows in parallel with downstream dependency
migration-guide ,  prefect-2-0
1 89 November 11, 2022
How can I specify the retry behavior for a specific task?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  retries ,  getting-started ,  task-decorator
6 356 November 10, 2022
How to use prefect for load balancing with dynamic parameters
getting-started
3 51 November 10, 2022
Is there an equivalent to sensors in Prefect? How do I trigger event-driven workflows?
backend ,  parameters ,  flow-of-flows ,  aws ,  event-driven ,  create_flow_run ,  signals ,  orchestrator-pattern ,  aws-lambda ,  sensors-orchestration-pattern
2 1279 November 4, 2022
How can I set task-level timeouts in Prefect 2?
prefect-2-0 ,  timeout ,  prefect-future ,  wait
3 305 November 1, 2022
How to resatrt (part of) the flow in prefect 2.0
prefect-2-0 ,  create_flow_run
3 56 November 1, 2022
Avoiding locked database with larger flows -- recommendations for something better than a delay?
prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner
7 122 October 27, 2022
Timeout error for long-running tasks running Prefect 2.0 in WSL
prefect-2-0 ,  timeout
12 874 October 22, 2022
Does Prefect support caching on a flow level rather than only on a task level? How to avoid task run recomputation of several tasks in a flow?
prefect-1-0 ,  caching ,  results ,  checkpointing
7 213 October 21, 2022
Can I set a custom name of a task or task run at run-time in Prefect 2.0?
prefect-2-0 ,  task-name ,  task-args ,  task_run_name
3 361 October 18, 2022
Using Prefect Client and Prefect Context for event driven workflows
prefect-2-0 ,  context ,  event-driven ,  orchestrator-pattern ,  api
0 180 September 29, 2022
Does Prefect 2.0 support filters and adjustments e.g. to restrict the schedule to only run during business hours?
migration-guide ,  prefect-2-0 ,  local-development ,  scheduling ,  interval-schedule ,  cron ,  unit-testing ,  rrule
0 246 April 29, 2022
How can I create a subflow?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  getting-started
0 668 January 27, 2022
How to pass data between subflows?
prefect-2-0 ,  subflows ,  data-dependencies
0 268 April 8, 2022
How can I attach a schedule to a flow?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  deployment ,  parameters ,  scheduling ,  getting-started ,  interval-schedule
4 455 August 19, 2022
Is it possible to skip a new scheduled flow run if a previous flow run of the same flow is still in progress?
prefect-1-0 ,  state_handlers ,  skipping-task-or-flow-runs
5 353 August 10, 2022
How to override default parameter values when creating a flow run from deployment?
prefect-2-0 ,  deployment ,  parameters ,  create_flow_run
5 194 August 8, 2022
How can I organize my flows based on my business logic or team structure?
organizational ,  migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  cloud-1-0 ,  deployment ,  tags ,  projects
0 669 January 25, 2022
How to take action on a state change of a task run? (task-level state handler)
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  automations ,  state_handlers ,  states ,  slack-notifications ,  notifications ,  failure ,  getting-started ,  email-notifications
11 1079 August 1, 2022
How to map over flows with various parameter values?
prefect-1-0 ,  parameters ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run ,  mapping
1 661 July 25, 2022
What is the default TaskRunner (executor)?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  executor ,  task-runner ,  defaults
1 588 July 21, 2022
What's the role of agents and work queues, and how the concept of agents differ between Prefect 1.0 and 2.0?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  agent ,  work-queue ,  work-queue-agent-relationship
0 504 April 5, 2022