Topics tagged dag-flow-structure

Topic	Replies	Views	Activity
Can I define my tasks as classes rather than functions? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , imperative-api , getting-started	6	1304	December 13, 2022
Can I call a task from another task? FAQ prefect-2-0 , dag-flow-structure , state-dependencies , task-decorator , task	0	561	August 21, 2022
Conditionally running tasks and filtering their results in Prefect 1.0 and 2.0 Tutorials & Resources migration-guide , dag-flow-structure	0	377	July 25, 2022
Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level? Workflow Orchestration migration-guide , prefect-2-0 , dag-flow-structure , subflows , getting-started , task-decorator , best-practices , flow-decorator	0	504	March 17, 2022
"With Case" statement + Results from task Usage Questions prefect-1-0 , dag-flow-structure	6	396	June 15, 2022
Best approach to create a library of reused declarative tasks? Usage Questions prefect-1-0 , dag-flow-structure	1	249	May 7, 2022
How to use Prefect together with an EMR cluster on AWS? Distributed Computing prefect-1-0 , dag-flow-structure , aws , resource-manager , emr , spark	0	520	February 7, 2022
How can I define state dependencies between tasks? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , state-dependencies , getting-started , task-decorator , wait_for , upstream_tasks , basics	1	968	April 11, 2022
How can I structure my flows (both in the repository and the UI) and set project-based permissions (RBAC)? FAQ organizational , prefect-1-0 , dag-flow-structure , auth-and-permissions , rbac , repository-structure	0	293	April 6, 2022
When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version? Workflow Orchestration prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash	0	467	March 2, 2022
How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run) Workflow Orchestration prefect-1-0 , dag-flow-structure , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , flow_run_name , task-decorator	0	602	March 11, 2022
Prefect 1.1.0 is out introducing pipe operator for functional chaining, Kubernetes authentication support to HashiCorp Vault, improving idempotency key for mapped subflows, fixing Hasura zombie processes and many task library enhancements! Announcements prefect-1-0 , dag-flow-structure , release-notes , auth-and-permissions , kubernetes , server , task-library , create_flow_run , airbyte , user-contribution , task-decorator , hasura , run-namespaced-job , cube-js	0	245	March 11, 2022
How to use PrefectSecret with (Azure) Task Library and what are the differences between secrets in a module vs. Flow scope? Integrations prefect-1-0 , dag-flow-structure , local-development , cloud-1-0 , secrets , azure , build-time-vs-runtime	2	248	February 24, 2022
How to sequentially loop several cleanup tasks over a set of files to process for ETL Workflow Orchestration prefect-1-0 , dag-flow-structure , local-development , elt , mapping , looping , sequential-processing , etl	2	422	February 24, 2022
How to avoid tasks such as List, Tuple or Dict in a flow structure? Workflow Orchestration prefect-1-0 , dag-flow-structure , state-dependencies , data-dependencies , collection_tasks	0	298	February 18, 2022
How can I define a flow? Getting Started migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , local-development , getting-started , basics	3	987	February 17, 2022
How to build conditional logic within mapped tasks? Workflow Orchestration prefect-1-0 , dag-flow-structure , parallel-processing , conditional-logic , control-flow , mapping , apply_map	0	515	February 16, 2022
When using the S3Upload task from the task library, I'm getting an error: "AttributeError: 'str' object has no attribute 'bucket'" - how to fix this? Integrations prefect-1-0 , dag-flow-structure , aws , s3 , imperative-api , task-library	0	293	February 14, 2022
Can I run acyclic workflows in Prefect (i.e. flows with cycles and loops)? Workflow Orchestration prefect-1-0 , dag-flow-structure , mapping , dynamicism , looping , dag	0	300	February 4, 2022
How can I send Slack alert on failure including dbt output? Integrations prefect-1-0 , dag-flow-structure , dbt , state_handlers , task-library , slack-notifications , notifications , failure , data-dependencies	1	1296	January 29, 2022

Can I define my tasks as classes rather than functions?

Workflow Orchestration

migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , imperative-api , getting-started

6

1304

December 13, 2022

Can I call a task from another task?

FAQ

prefect-2-0 , dag-flow-structure , state-dependencies , task-decorator , task

0

561

August 21, 2022

Conditionally running tasks and filtering their results in Prefect 1.0 and 2.0

Tutorials & Resources

migration-guide , dag-flow-structure

0

377

July 25, 2022

Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level?

Workflow Orchestration

migration-guide , prefect-2-0 , dag-flow-structure , subflows , getting-started , task-decorator , best-practices , flow-decorator

0

504

March 17, 2022

"With Case" statement + Results from task

Usage Questions

prefect-1-0 , dag-flow-structure

6

396

June 15, 2022

Best approach to create a library of reused declarative tasks?

Usage Questions

prefect-1-0 , dag-flow-structure

1

249

May 7, 2022

How to use Prefect together with an EMR cluster on AWS?

Distributed Computing

prefect-1-0 , dag-flow-structure , aws , resource-manager , emr , spark

0

520

February 7, 2022

How can I define state dependencies between tasks?

Workflow Orchestration

migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , state-dependencies , getting-started , task-decorator , wait_for , upstream_tasks , basics

1

968

April 11, 2022

How can I structure my flows (both in the repository and the UI) and set project-based permissions (RBAC)?

FAQ

organizational , prefect-1-0 , dag-flow-structure , auth-and-permissions , rbac , repository-structure

0

293

April 6, 2022

When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version?

Workflow Orchestration

prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash

0

467

March 2, 2022

How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run)

Workflow Orchestration

prefect-1-0 , dag-flow-structure , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , flow_run_name , task-decorator

0

602

March 11, 2022

Prefect 1.1.0 is out introducing pipe operator for functional chaining, Kubernetes authentication support to HashiCorp Vault, improving idempotency key for mapped subflows, fixing Hasura zombie processes and many task library enhancements!

Announcements

prefect-1-0 , dag-flow-structure , release-notes , auth-and-permissions , kubernetes , server , task-library , create_flow_run , airbyte , user-contribution , task-decorator , hasura , run-namespaced-job , cube-js

0

245

March 11, 2022

How to use PrefectSecret with (Azure) Task Library and what are the differences between secrets in a module vs. Flow scope?

Integrations

prefect-1-0 , dag-flow-structure , local-development , cloud-1-0 , secrets , azure , build-time-vs-runtime