task-runner

Topic Replies Views Activity
Flow Runner vs. Task Runner - what are the differences? when to use which one and how?
DataOps Tutorials for Prefect 2.0
prefect-2-0 ,  flow-runner ,  task-runner ,  todo
0 18 June 19, 2022
How to decide when to use the default ConcurrentTaskRunner vs. SequentialTaskRunner vs. DaskTaskRunner vs. RayTaskRunner? What are performance benchmarks for given use cases?
DataOps Tutorials for Prefect 2.0
task-runner ,  dask-task-runner ,  async ,  ray-task-runner ,  todo ,  concurrent-task-runner
0 13 June 19, 2022
New Prefect 2.0 release: 2.0b4 including flow and task runner improvements, performance optimizations, numerous fixes and new Prefect Collections!
Announcements
prefect-2-0 ,  release-notes ,  cli ,  storage ,  deployment ,  flow-runner ,  task-runner ,  dask-task-runner ,  async ,  twitter ,  kubernetes-flow-runner ,  great-expectations ,  ray-task-runner ,  cube-js ,  prefect-collections ,  subprocess-flow-runner ,  github
2 190 May 13, 2022
How can I run tasks in parallel?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  infrastructure ,  multithreading ,  multiprocessing ,  basics
0 110 January 31, 2022
Why is my function not running in parallel even with DaskTaskRunner / RayTaskRunner?
Distributed Computing
prefect-2-0 ,  dask ,  task-runner ,  parallel-processing ,  dask-task-runner ,  dask-annotations
13 129 April 13, 2022
How can I parallelize execution across 8 CPU cores?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  local-dask-executor ,  infrastructure ,  multithreading ,  multiprocessing
0 133 January 31, 2022
What is the difference between Prefect and Celery?
FAQ
backend ,  executor ,  task-runner ,  celery
0 55 March 21, 2022
New Orion release 2.0a13 featuring work queues, agents, profiles, remote object storage and distributed parallel processing with Ray!
Announcements
prefect-2-0 ,  release-notes ,  storage ,  aws ,  s3 ,  gcp ,  task-runner ,  parallel-processing ,  agent ,  azure ,  concurrency-limits ,  kubernetes-flow-runner ,  filters ,  gcs ,  docker-flow-runner ,  profiles-prefect-config ,  work-queue ,  ray ,  ray-task-runner ,  azure-blob-storage
4 241 March 7, 2022
Can my subflow use a different task runner than my parent flow?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  executor ,  task-runner ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run
0 100 January 31, 2022
How can I configure a specific Dask cluster class?
Distributed Computing
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  aws ,  dask ,  executor ,  task-runner ,  parallel-processing ,  fargate ,  dask-task-runner ,  dask-cloud-provider ,  docker-image
0 91 January 21, 2022
What is the default TaskRunner (executor)?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  executor ,  task-runner ,  defaults
0 66 January 25, 2022
How can I configure my flow to run with Dask?
Distributed Computing
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  getting-started
1 192 February 25, 2022
New Orion release 2.0a10 including concurrency limits, ConcurrentTaskRunner, CLI refinements and improved database migrations
Announcements
prefect-2-0 ,  release-notes ,  cli ,  parameters ,  task-runner ,  parallel-processing ,  concurrency-limits ,  async ,  database-migrations
0 127 February 9, 2022