dask-task-runner

Topic Replies Views Activity
How to decide when to use the default ConcurrentTaskRunner vs. SequentialTaskRunner vs. DaskTaskRunner vs. RayTaskRunner? What are performance benchmarks for given use cases?
DataOps Tutorials for Prefect 2.0
task-runner ,  dask-task-runner ,  async ,  ray-task-runner ,  todo ,  concurrent-task-runner
0 13 June 19, 2022
How to scale out Prefect 2.0 flows with Dask on AWS ECS Fargate using FargateCluster from dask_cloudprovider
DataOps Tutorials for Prefect 2.0
prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner ,  dask-cloud-provider ,  todo
0 13 June 19, 2022
How to scale out Prefect 2.0 flows with Dask on Kubernetes using KubeCluster
DataOps Tutorials for Prefect 2.0
prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner ,  kube-cluster ,  todo
0 10 June 19, 2022
How to scale out Prefect 2.0 flows with Dask on Coiled
DataOps Tutorials for Prefect 2.0
prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner ,  coiled ,  todo
0 13 June 19, 2022
Why are the Dask logs not being shown in the Prefect UI?
Distributed Computing
prefect-1-0 ,  prefect-2-0 ,  logging ,  kubernetes ,  dask ,  ui ,  dask-task-runner
3 185 June 21, 2022
How can I map over a list of inputs?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner ,  mapping ,  local-dask-executor ,  getting-started ,  to-review
5 139 May 16, 2022
New Prefect 2.0 release: 2.0b4 including flow and task runner improvements, performance optimizations, numerous fixes and new Prefect Collections!
Announcements
prefect-2-0 ,  release-notes ,  cli ,  storage ,  deployment ,  flow-runner ,  task-runner ,  dask-task-runner ,  async ,  twitter ,  kubernetes-flow-runner ,  great-expectations ,  ray-task-runner ,  cube-js ,  prefect-collections ,  subprocess-flow-runner ,  github
2 190 May 13, 2022
PicklingError error when using a DaskTaskRunner
Distributed Computing
prefect-2-0 ,  pickle ,  dask-task-runner
0 39 May 12, 2022
How can I run tasks in parallel?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  infrastructure ,  multithreading ,  multiprocessing ,  basics
0 110 January 31, 2022
How can I force memory to be released at the end of a dask task?
Distributed Computing
prefect-2-0 ,  dask ,  dask-task-runner ,  cpu-memory-resource-allocation
3 75 April 22, 2022
Why is my function not running in parallel even with DaskTaskRunner / RayTaskRunner?
Distributed Computing
prefect-2-0 ,  dask ,  task-runner ,  parallel-processing ,  dask-task-runner ,  dask-annotations
13 129 April 13, 2022
How can I parallelize execution across 8 CPU cores?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  local-dask-executor ,  infrastructure ,  multithreading ,  multiprocessing
0 133 January 31, 2022
Prefect 2.0 UI doesn't seem to show all task runs running on Dask in real-time
Distributed Computing
prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner
19 79 March 29, 2022
How can I configure a specific Dask cluster class?
Distributed Computing
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  aws ,  dask ,  executor ,  task-runner ,  parallel-processing ,  fargate ,  dask-task-runner ,  dask-cloud-provider ,  docker-image
0 91 January 21, 2022
How can I configure my flow to run with Dask?
Distributed Computing
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  getting-started
1 192 February 25, 2022