Topics tagged flow-of-flows

Topic	Replies	Views	Activity
Can prefect run each subflow in different machine? Help prefect-2-0 , deployment , flow-of-flows , task-runner , subflows , infrastructure	0	45	June 25, 2024
How can I run multiple subflows (or child flows) in parallel? Help migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , parallel-processing , create_flow_run , orchestrator-pattern , subflows , async , mapping , local-dask-executor , marvin	25	11308	April 20, 2024
Subflows logs do not appear in Parent Flow Help prefect-2-0 , logging , flow-of-flows , subflows	1	504	November 1, 2023
Not possible to share data between subflows Help prefect-2-0 , flow-of-flows , results , subflows	1	416	July 20, 2023
How to handle custom resource (CPU vs. GPU) and code dependencies across various tasks and flows? Archive prefect-1-0 , prefect-2-0 , flow-of-flows , kubernetes , aws , orchestrator-pattern , subflows , infrastructure , cpu_request , kubernetes-agent , aws-eks , radar-view , gpu-intensive-workflows , cpu-memory-resource-allocation	3	3144	July 17, 2023
API for finding child flows of a parent flow programmatically? Help prefect-2-0 , flow-of-flows	0	385	March 1, 2023
Can Prefect run each task in a different Docker container? Archive prefect-1-0 , flow-of-flows , docker , create_flow_run , orchestrator-pattern , wait_for_flow_run , infrastructure , code-dependencies , docker-run , docker-image , base-image	4	3269	February 22, 2023
How to create a flow run from deployment (orchestrator pattern)? Archive prefect-2-0 , deployment , flow-of-flows , create_flow_run , orchestrator-pattern , create_flow_run_from_deployment	11	5420	February 9, 2023
Is there an equivalent to sensors in Prefect? How do I trigger event-driven workflows? Archive backend , parameters , flow-of-flows , aws , event-driven , create_flow_run , signals , orchestrator-pattern , aws-lambda , sensors-orchestration-pattern	2	4503	November 4, 2022
How can I create a subflow? Archive migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , getting-started	0	2624	January 27, 2022
Migrating to Prefect 2.0: From Flow of Flows to Subflows Archive migration-guide , flow-of-flows , subflows	0	4590	August 3, 2022
How to map over flows with various parameter values? Archive prefect-1-0 , parameters , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , mapping	1	2717	July 25, 2022
Best way to create a DAG with tasks that should occur repeatedly at different time intervals Archive prefect-1-0 , prefect-2-0 , flow-of-flows , scheduling , subflows	9	1669	June 17, 2022
How to fix the error: Failed to load and execute Flow's environment: TypeError("got an unexpected keyword argument 'raise_final_state'") Archive prefect-1-0 , flow-of-flows , create_flow_run , wait_for_flow_run	0	1516	April 27, 2022
How can I create a subflow and block until it’s completed? Archive migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , to-review , start_flow_run	0	2676	January 28, 2022
How to schedule and orchestrate a flow-of-flows for a data warehousing (ETL) use case? Archive prefect-1-0 , flow-of-flows , scheduling , create_flow_run , orchestrator-pattern , wait_for_flow_run	0	1902	April 5, 2022
How do I restart a child flow run in a flow-of-flows? Archive prefect-1-0 , flow-of-flows , create_flow_run , wait_for_flow_run , idempotency-key , restarts	0	1035	March 31, 2022
Can I run a flow-of-flows that triggers each child flow on a different machine? Archive prefect-1-0 , flow-of-flows , agent , labels , create_flow_run , orchestrator-pattern , wait_for_flow_run , infrastructure	0	1489	February 7, 2022
Task-specific environment/memory requirements Archive prefect-1-0 , flow-of-flows , orchestrator-pattern , memory_request , cpu_request , docker-run , cpu-memory-resource-allocation	2	1357	March 26, 2022
How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run) Archive prefect-1-0 , dag-flow-structure , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , flow_run_name , task-decorator	0	1304	March 11, 2022
Can my subflow use a different task runner than my parent flow? Archive migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , executor , task-runner , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run	0	1064	January 31, 2022
How to build a parametrized flow of flows using GCS storage and Docker agent? Archive prefect-1-0 , parameters , flow-of-flows , gcp , docker , orchestrator-pattern , subflows , data-dependencies , local-agent , docker-agent , docker-run	0	605	February 26, 2022
How to pass data from a child flow to another task in a parent flow? Archive prefect-1-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , data-dependencies , get_task_run_result	0	1802	January 31, 2022
How to fix the error: "ValueError: Could not infer an active Flow context while creating edge to <Task: create_flow_run>" Archive prefect-1-0 , flow-of-flows , gcp , secrets , orchestrator-pattern , subflows , graphql , api-payload-limits , gcs	0	1463	February 18, 2022
How to build a conditional flow-of-flows, i.e. trigger a different child flow depending on the upstream flow's state? Archive prefect-1-0 , flow-of-flows , states , reference-tasks , conditional-logic , state-dependencies , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , triggers	0	1958	February 7, 2022
How to dynamically change the run_name in the create_flow_run task? Archive prefect-1-0 , parameters , flow-of-flows , create_flow_run , orchestrator-pattern , dynamicism , flow_run_name	0	1584	January 28, 2022
How can I navigate to the child flow run from a parent flow? Archive prefect-1-0 , flow-of-flows , orchestrator-pattern , subflows	0	673	February 4, 2022

Can prefect run each subflow in different machine?

Help

prefect-2-0 , deployment , flow-of-flows , task-runner , subflows , infrastructure

0

45

June 25, 2024

How can I run multiple subflows (or child flows) in parallel?

Help

migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , parallel-processing , create_flow_run , orchestrator-pattern , subflows , async , mapping , local-dask-executor , marvin

25

11308

April 20, 2024

Subflows logs do not appear in Parent Flow

Help

prefect-2-0 , logging , flow-of-flows , subflows

1

504

November 1, 2023

Not possible to share data between subflows

Help

prefect-2-0 , flow-of-flows , results , subflows

1

416

July 20, 2023

How to handle custom resource (CPU vs. GPU) and code dependencies across various tasks and flows?

Archive

prefect-1-0 , prefect-2-0 , flow-of-flows , kubernetes , aws , orchestrator-pattern , subflows , infrastructure , cpu_request , kubernetes-agent , aws-eks , radar-view , gpu-intensive-workflows , cpu-memory-resource-allocation

3

3144

July 17, 2023

API for finding child flows of a parent flow programmatically?

Help

prefect-2-0 , flow-of-flows

0

385

March 1, 2023

Can Prefect run each task in a different Docker container?

Archive

prefect-1-0 , flow-of-flows , docker , create_flow_run , orchestrator-pattern , wait_for_flow_run , infrastructure , code-dependencies , docker-run , docker-image , base-image

4

3269

February 22, 2023

How to create a flow run from deployment (orchestrator pattern)?

Archive

prefect-2-0 , deployment , flow-of-flows , create_flow_run , orchestrator-pattern , create_flow_run_from_deployment

11

5420

February 9, 2023

Is there an equivalent to sensors in Prefect? How do I trigger event-driven workflows?

Archive

backend , parameters , flow-of-flows , aws , event-driven , create_flow_run , signals , orchestrator-pattern , aws-lambda , sensors-orchestration-pattern

2

4503

November 4, 2022

How can I create a subflow?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , getting-started

0

2624

January 27, 2022

Migrating to Prefect 2.0: From Flow of Flows to Subflows

Archive

migration-guide , flow-of-flows , subflows

0

4590

August 3, 2022

How to map over flows with various parameter values?

Archive

prefect-1-0 , parameters , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , mapping

1

2717

July 25, 2022

Best way to create a DAG with tasks that should occur repeatedly at different time intervals

Archive

prefect-1-0 , prefect-2-0 , flow-of-flows , scheduling , subflows

9

1669

June 17, 2022

How to fix the error: Failed to load and execute Flow's environment: TypeError("got an unexpected keyword argument 'raise_final_state'")

Archive

prefect-1-0 , flow-of-flows , create_flow_run , wait_for_flow_run

0

1516

April 27, 2022

How can I create a subflow and block until it’s completed?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , to-review , start_flow_run

0

2676

January 28, 2022

How to schedule and orchestrate a flow-of-flows for a data warehousing (ETL) use case?

Archive

prefect-1-0 , flow-of-flows , scheduling , create_flow_run , orchestrator-pattern , wait_for_flow_run

0

1902

April 5, 2022

How do I restart a child flow run in a flow-of-flows?

Archive

prefect-1-0 , flow-of-flows , create_flow_run , wait_for_flow_run , idempotency-key , restarts

0

1035

March 31, 2022

Can I run a flow-of-flows that triggers each child flow on a different machine?

Archive

prefect-1-0 , flow-of-flows , agent , labels , create_flow_run , orchestrator-pattern , wait_for_flow_run , infrastructure

0

1489

February 7, 2022

Task-specific environment/memory requirements

Archive

prefect-1-0 , flow-of-flows , orchestrator-pattern , memory_request , cpu_request , docker-run , cpu-memory-resource-allocation

2

1357

March 26, 2022

How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run)

Archive

prefect-1-0 , dag-flow-structure , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , flow_run_name , task-decorator

0

1304

March 11, 2022

Can my subflow use a different task runner than my parent flow?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , flow-of-flows , executor , task-runner , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run

0

1064

January 31, 2022

How to build a parametrized flow of flows using GCS storage and Docker agent?

Archive

prefect-1-0 , parameters , flow-of-flows , gcp , docker , orchestrator-pattern , subflows , data-dependencies , local-agent , docker-agent , docker-run

0

605

February 26, 2022

How to pass data from a child flow to another task in a parent flow?

Archive

prefect-1-0 , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , data-dependencies , get_task_run_result

0

1802

January 31, 2022

How to fix the error: "ValueError: Could not infer an active Flow context while creating edge to <Task: create_flow_run>"

Archive

prefect-1-0 , flow-of-flows , gcp , secrets , orchestrator-pattern , subflows , graphql , api-payload-limits , gcs

0

1463

February 18, 2022

How to build a conditional flow-of-flows, i.e. trigger a different child flow depending on the upstream flow's state?

Archive

prefect-1-0 , flow-of-flows , states , reference-tasks , conditional-logic , state-dependencies , create_flow_run , orchestrator-pattern , subflows , wait_for_flow_run , triggers

0

1958

February 7, 2022

How to dynamically change the run_name in the create_flow_run task?

Archive

prefect-1-0 , parameters , flow-of-flows , create_flow_run , orchestrator-pattern , dynamicism , flow_run_name

0

1584

January 28, 2022

How can I navigate to the child flow run from a parent flow?

Archive

prefect-1-0 , flow-of-flows , orchestrator-pattern , subflows

0

673

February 4, 2022