orchestrator-pattern

Topic Replies Views Activity
How to orchestrate ELT with Prefect 2.0, dbt, & Snowflake
DataOps Tutorials for Prefect 2.0
prefect-2-0 ,  snowflake ,  dbt ,  orchestrator-pattern ,  subflows ,  modern-data-stack ,  todo
0 19 June 19, 2022
How to use subflows and orchestrator pattern for Data Warehouse use cases?
DataOps Tutorials for Prefect 2.0
prefect-2-0 ,  orchestrator-pattern ,  subflows ,  modern-data-stack ,  todo ,  poc
0 20 June 19, 2022
Can Prefect run each task in a different Docker container?
Deployments & DevOps
prefect-1-0 ,  flow-of-flows ,  docker ,  create_flow_run ,  orchestrator-pattern ,  wait_for_flow_run ,  infrastructure ,  code-dependencies ,  docker-run ,  docker-image ,  base-image
1 339 June 14, 2022
How can I run multiple subflows (or child flows) in parallel?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  parallel-processing ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  async ,  mapping ,  local-dask-executor
3 191 May 31, 2022
How can I create a subflow?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  getting-started
0 218 January 27, 2022
How to create a flow run from deployment (orchestrator pattern)?
Workflow Orchestration
prefect-2-0 ,  deployment ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  create_flow_run_from_deployment
8 158 April 27, 2022
How can I create a subflow and block until it’s completed?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run ,  to-review ,  start_flow_run
0 237 January 28, 2022
How to schedule and orchestrate a flow-of-flows for a data warehousing (ETL) use case?
Workflow Orchestration
prefect-1-0 ,  flow-of-flows ,  scheduling ,  create_flow_run ,  orchestrator-pattern ,  wait_for_flow_run
0 201 April 5, 2022
Can I run a flow-of-flows that triggers each child flow on a different machine?
Deployments & DevOps
prefect-1-0 ,  flow-of-flows ,  agent ,  labels ,  create_flow_run ,  orchestrator-pattern ,  wait_for_flow_run ,  infrastructure
0 222 February 7, 2022
Task-specific environment/memory requirements
Deployments & DevOps
prefect-1-0 ,  flow-of-flows ,  orchestrator-pattern ,  memory_request ,  cpu_request ,  docker-run ,  cpu-memory-resource-allocation
2 94 March 26, 2022
How to handle custom resource (CPU vs. GPU) and code dependencies across various tasks and flows?
Workflow Orchestration
prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  kubernetes ,  aws ,  orchestrator-pattern ,  subflows ,  infrastructure ,  cpu_request ,  kubernetes-agent ,  aws-eks ,  radar-view ,  gpu-intensive-workflows ,  cpu-memory-resource-allocation
2 267 March 15, 2022
Is there an equivalent to sensors in Prefect? How do I trigger event-driven workflows?
Workflow Orchestration
backend ,  parameters ,  flow-of-flows ,  aws ,  event-driven ,  create_flow_run ,  signals ,  orchestrator-pattern ,  aws-lambda ,  sensors-orchestration-pattern
0 430 January 26, 2022
How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run)
Workflow Orchestration
prefect-1-0 ,  dag-flow-structure ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  flow_run_name ,  task-decorator
0 91 March 11, 2022
Can my subflow use a different task runner than my parent flow?
Workflow Orchestration
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  executor ,  task-runner ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run
0 100 January 31, 2022
How to build a parametrized flow of flows using GCS storage and Docker agent?
Deployments & DevOps
prefect-1-0 ,  parameters ,  flow-of-flows ,  gcp ,  docker ,  orchestrator-pattern ,  subflows ,  data-dependencies ,  local-agent ,  docker-agent ,  docker-run
0 87 February 26, 2022
How to pass data from a child flow to another task in a parent flow?
Workflow Orchestration
prefect-1-0 ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  data-dependencies ,  get_task_run_result
0 147 January 31, 2022
How to map over flows with various parameter values?
Workflow Orchestration
prefect-1-0 ,  parameters ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run ,  mapping
0 133 February 23, 2022
How to fix the error: "ValueError: Could not infer an active Flow context while creating edge to <Task: create_flow_run>"
Workflow Orchestration
prefect-1-0 ,  flow-of-flows ,  gcp ,  secrets ,  orchestrator-pattern ,  subflows ,  graphql ,  api-payload-limits ,  gcs
0 164 February 18, 2022
How to build a conditional flow-of-flows, i.e. trigger a different child flow depending on the upstream flow's state?
Workflow Orchestration
prefect-1-0 ,  flow-of-flows ,  states ,  reference-tasks ,  conditional-logic ,  state-dependencies ,  create_flow_run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run ,  triggers
0 168 February 7, 2022
How to dynamically change the run_name in the create_flow_run task?
Workflow Orchestration
prefect-1-0 ,  parameters ,  flow-of-flows ,  create_flow_run ,  orchestrator-pattern ,  dynamicism ,  flow_run_name
0 231 January 28, 2022
How can I navigate to the child flow run from a parent flow?
UI
prefect-1-0 ,  flow-of-flows ,  orchestrator-pattern ,  subflows
0 95 February 4, 2022