Topics tagged dag-flow-structure

Topic	Replies	Views	Activity
How can I define state dependencies between tasks? Archive migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , state-dependencies , getting-started , task-decorator , wait_for , upstream_tasks , basics	2	3644	May 28, 2024
How can I send Slack alert on failure including dbt output? Archive prefect-1-0 , dag-flow-structure , dbt , state_handlers , task-library , slack-notifications , notifications , failure , data-dependencies	7	3320	May 17, 2023
Can I define my tasks as classes rather than functions? Archive migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , imperative-api , getting-started	6	4539	December 13, 2022
Can I call a task from another task? Archive prefect-2-0 , dag-flow-structure , state-dependencies , task-decorator , task	0	2385	August 21, 2022
Conditionally running tasks and filtering their results in Prefect 1.0 and 2.0 Archive migration-guide , dag-flow-structure	0	844	July 25, 2022
Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level? Archive migration-guide , prefect-2-0 , dag-flow-structure , subflows , getting-started , task-decorator , best-practices , flow-decorator	0	1999	March 17, 2022
"With Case" statement + Results from task Archive prefect-1-0 , dag-flow-structure	6	1502	June 15, 2022
Best approach to create a library of reused declarative tasks? Archive prefect-1-0 , dag-flow-structure	1	804	May 7, 2022
How to use Prefect together with an EMR cluster on AWS? Archive prefect-1-0 , dag-flow-structure , aws , resource-manager , emr , spark	0	1628	February 7, 2022
How can I structure my flows (both in the repository and the UI) and set project-based permissions (RBAC)? Archive organizational , prefect-1-0 , dag-flow-structure , auth-and-permissions , rbac , repository-structure	0	955	April 6, 2022
When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version? Archive prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash	0	1354	March 2, 2022
How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run) Archive prefect-1-0 , dag-flow-structure , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , flow_run_name , task-decorator	0	1307	March 11, 2022
Prefect 1.1.0 is out introducing pipe operator for functional chaining, Kubernetes authentication support to HashiCorp Vault, improving idempotency key for mapped subflows, fixing Hasura zombie processes and many task library enhancements! Announcements prefect-1-0 , dag-flow-structure , release-notes , auth-and-permissions , kubernetes , server , task-library , create_flow_run , airbyte , user-contribution , task-decorator , hasura , run-namespaced-job , cube-js	0	562	March 11, 2022
How to use PrefectSecret with (Azure) Task Library and what are the differences between secrets in a module vs. Flow scope? Archive prefect-1-0 , dag-flow-structure , local-development , cloud-1-0 , secrets , azure , build-time-vs-runtime	2	602	February 24, 2022
How to sequentially loop several cleanup tasks over a set of files to process for ETL Archive prefect-1-0 , dag-flow-structure , local-development , elt , mapping , looping , sequential-processing , etl	2	1380	February 24, 2022
How to avoid tasks such as List, Tuple or Dict in a flow structure? Archive prefect-1-0 , dag-flow-structure , state-dependencies , data-dependencies , collection_tasks	0	717	February 18, 2022
How can I define a flow? Help migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , local-development , getting-started , basics	3	1768	February 17, 2022
How to build conditional logic within mapped tasks? Archive prefect-1-0 , dag-flow-structure , parallel-processing , conditional-logic , control-flow , mapping , apply_map	0	1171	February 16, 2022
When using the S3Upload task from the task library, I'm getting an error: "AttributeError: 'str' object has no attribute 'bucket'" - how to fix this? Archive prefect-1-0 , dag-flow-structure , aws , s3 , imperative-api , task-library	0	785	February 14, 2022
Can I run acyclic workflows in Prefect (i.e. flows with cycles and loops)? Archive prefect-1-0 , dag-flow-structure , mapping , dynamicism , looping , dag	0	724	February 4, 2022

How can I define state dependencies between tasks?

migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , state-dependencies , getting-started , task-decorator , wait_for , upstream_tasks , basics

2

3644

May 28, 2024

How can I send Slack alert on failure including dbt output?

Archive

prefect-1-0 , dag-flow-structure , dbt , state_handlers , task-library , slack-notifications , notifications , failure , data-dependencies

7

3320

May 17, 2023

Can I define my tasks as classes rather than functions?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , dag-flow-structure , imperative-api , getting-started

6

4539

December 13, 2022

Can I call a task from another task?

Archive

prefect-2-0 , dag-flow-structure , state-dependencies , task-decorator , task

0

2385

August 21, 2022

Conditionally running tasks and filtering their results in Prefect 1.0 and 2.0

Archive

migration-guide , dag-flow-structure

0

844

July 25, 2022

Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level?

Archive

migration-guide , prefect-2-0 , dag-flow-structure , subflows , getting-started , task-decorator , best-practices , flow-decorator

0

1999

March 17, 2022

"With Case" statement + Results from task

Archive

prefect-1-0 , dag-flow-structure

6

1502

June 15, 2022

Best approach to create a library of reused declarative tasks?

Archive

prefect-1-0 , dag-flow-structure

1

804

May 7, 2022

How to use Prefect together with an EMR cluster on AWS?

Archive

prefect-1-0 , dag-flow-structure , aws , resource-manager , emr , spark

0

1628

February 7, 2022

How can I structure my flows (both in the repository and the UI) and set project-based permissions (RBAC)?

Archive

organizational , prefect-1-0 , dag-flow-structure , auth-and-permissions , rbac , repository-structure

0

955

April 6, 2022

When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version?

Archive

prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash

0

1354

March 2, 2022

How to customize how the name of a task triggering a child flow run is displayed in the parent DAG? (i.e. how to override the default task name create_flow_run)

Archive

prefect-1-0 , dag-flow-structure , flow-of-flows , create_flow_run , orchestrator-pattern , subflows , flow_run_name , task-decorator

0

1307

March 11, 2022

Prefect 1.1.0 is out introducing pipe operator for functional chaining, Kubernetes authentication support to HashiCorp Vault, improving idempotency key for mapped subflows, fixing Hasura zombie processes and many task library enhancements!

Announcements

prefect-1-0 , dag-flow-structure , release-notes , auth-and-permissions , kubernetes , server , task-library , create_flow_run , airbyte , user-contribution , task-decorator , hasura , run-namespaced-job , cube-js

0

562

March 11, 2022

How to use PrefectSecret with (Azure) Task Library and what are the differences between secrets in a module vs. Flow scope?

Archive

prefect-1-0 , dag-flow-structure , local-development , cloud-1-0 , secrets , azure , build-time-vs-runtime