Topics tagged executor

Topic	Replies	Views	Activity
What is the default TaskRunner (executor)? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , executor , task-runner , defaults	1	934	July 21, 2022
How can I configure my flow to run with Dask? Distributed Computing migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , getting-started	2	981	July 5, 2022
How can I configure a specific Dask cluster class? Distributed Computing migration-guide , prefect-1-0 , prefect-2-0 , aws , dask , executor , task-runner , parallel-processing , fargate , dask-task-runner , dask-cloud-provider , docker-image	0	351	January 21, 2022
How can I parallelize execution across 8 CPU cores? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , local-dask-executor , infrastructure , multithreading , multiprocessing	0	659	January 31, 2022
How can I run tasks in parallel? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , infrastructure , multithreading , multiprocessing , basics	0	890	January 31, 2022
How can I change the number of Dask workers in a DaskExecutor based on a custom parameter value? Distributed Computing prefect-1-0 , parameters , dask , executor , dynamicism	1	361	April 12, 2022
How to resume mapped task runs from failure at scale, or limit the amount of allowed runs that may fail? Workflow Orchestration prefect-1-0 , executor , retries , concurrency-limits , mapping , failure , restarts	0	268	April 7, 2022
What is the difference between Prefect and Celery? FAQ backend , executor , task-runner , celery	0	314	March 21, 2022
When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version? Workflow Orchestration prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash	0	450	March 2, 2022
When using LocalDaskExecutor, I got an error with Exit Code:: 139 - what does it mean? FAQ prefect-1-0 , executor , local-dask-executor , oom-error	0	456	March 2, 2022
Can my subflow use a different task runner than my parent flow? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , executor , task-runner , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run	0	381	January 31, 2022
Scale your Prefect + Dask Workflows to the Cloud - by Richard Pelgrim Distributed Computing prefect-1-0 , resource-manager , blog-post , dask , executor , parallel-processing , user-contribution , mapping , coiled , dask-executor	0	268	February 25, 2022
What's the relationship between the agent, executor, storage and FlowRunner and how do they affect local vs. backend-based flow run execution? Workflow Orchestration prefect-1-0 , local-development , backend , flow-runner , executor , agent	0	328	February 10, 2022
How to allocate more memory or more worker nodes on a per flow run basis? Distributed Computing prefect-1-0 , storage , run_config , kubernetes , dask , executor , kubernetes-run , coiled , dask-executor , memory_request , cpu_request	0	339	January 31, 2022
Scaling out with Prefect Tutorials & Resources prefect-1-0 , blog-post , dask , vertex , gcp , executor , parallel-processing , local-dask-executor	0	222	January 21, 2022

What is the default TaskRunner (executor)?

Workflow Orchestration

migration-guide , prefect-1-0 , prefect-2-0 , executor , task-runner , defaults

1

934

July 21, 2022

How can I configure my flow to run with Dask?

Distributed Computing

migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , getting-started

2

981

July 5, 2022

How can I configure a specific Dask cluster class?

Distributed Computing

migration-guide , prefect-1-0 , prefect-2-0 , aws , dask , executor , task-runner , parallel-processing , fargate , dask-task-runner , dask-cloud-provider , docker-image

0

351

January 21, 2022

How can I parallelize execution across 8 CPU cores?

Workflow Orchestration

migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , local-dask-executor , infrastructure , multithreading , multiprocessing

0

659

January 31, 2022

How can I run tasks in parallel?

Workflow Orchestration

migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , infrastructure , multithreading , multiprocessing , basics

0

890

January 31, 2022

How can I change the number of Dask workers in a DaskExecutor based on a custom parameter value?

Distributed Computing

prefect-1-0 , parameters , dask , executor , dynamicism

1

361

April 12, 2022

How to resume mapped task runs from failure at scale, or limit the amount of allowed runs that may fail?

Workflow Orchestration

prefect-1-0 , executor , retries , concurrency-limits , mapping , failure , restarts

0

268

April 7, 2022

What is the difference between Prefect and Celery?

FAQ

backend , executor , task-runner , celery

0

314

March 21, 2022

When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version?

Workflow Orchestration

prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash

0

450

March 2, 2022

When using LocalDaskExecutor, I got an error with Exit Code:: 139 - what does it mean?

FAQ

prefect-1-0 , executor , local-dask-executor , oom-error

0

456

March 2, 2022

Can my subflow use a different task runner than my parent flow?

Workflow Orchestration

migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , executor , task-runner , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run

0

381

January 31, 2022

Scale your Prefect + Dask Workflows to the Cloud - by Richard Pelgrim

Distributed Computing

prefect-1-0 , resource-manager , blog-post , dask , executor , parallel-processing , user-contribution , mapping , coiled , dask-executor

0

268

February 25, 2022

What's the relationship between the agent, executor, storage and FlowRunner and how do they affect local vs. backend-based flow run execution?

Workflow Orchestration

prefect-1-0 , local-development , backend , flow-runner , executor , agent