Workflow Orchestration


Topic Replies Views Activity
About the Workflow Orchestration category
0 63 January 21, 2022
Upon Flow failure, use an Automation to start another Flow
prefect-1-0 ,  automations
0 9 May 24, 2022
How to configure retries on a task from a Prefect Collection (in Prefect 2.0)?
prefect-2-0 ,  shell ,  retries ,  prefect-collections ,  with_options ,  prefect_shell
0 9 May 22, 2022
Are there any guidelines when to use flow vs task decorator? Which part of my workflow should be defined at a task, subflow vs. flow-level?
migration-guide ,  prefect-2-0 ,  flow-structure ,  subflows ,  getting-started ,  task-decorator ,  best-practices ,  flow-decorator
0 92 March 17, 2022
When should I use FileStorageBlock vs. specific cloud-platform storage block such as S3StorageBlock, GoogleCloudStorageBlock or AzureBlobStorageBlock?
prefect-2-0 ,  storage ,  s3 ,  gcs ,  best-practices ,  azure-blob-storage ,  s3-storage ,  blocks
0 20 May 18, 2022
Cloud - agent security
prefect-1-0
1 7 May 18, 2022
How to ensure that exceptions and errors are logged with ERROR log level rather than INFO?
prefect-1-0 ,  logging ,  signals ,  exception-handling ,  log-level
0 15 May 16, 2022
How can I map over a list of inputs?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  parallel-processing ,  dask-task-runner ,  mapping ,  local-dask-executor ,  getting-started ,  to-review
5 104 May 16, 2022
How can I create a subflow?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  create-flow-run ,  orchestrator-pattern ,  subflows ,  getting-started
0 139 January 27, 2022
How can I impose concurrency limits for flow and task runs?
organizational ,  migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  cloud-1-0 ,  concurrency-limits ,  to-review
0 75 January 27, 2022
Timeout error for long-running tasks running Prefect 2.0 in WSL
prefect-2-0 ,  timeout
8 99 May 12, 2022
What are best practices to build unit tests and integration tests for Prefect flows and for data pipelines in general?
prefect-1-0 ,  prefect-2-0 ,  best-practices ,  data-quality ,  data-validation ,  unit-testing
0 38 May 10, 2022
How can I test flows and tasks in Prefect 2.0?
prefect-2-0 ,  unit-testing
2 57 May 5, 2022
Why results need to be configured in order to use retries?
prefect-1-0 ,  retries ,  results
0 24 May 3, 2022
How to rerun all flows in a project?
prefect-1-0 ,  projects ,  create-flow-run ,  graphql
0 21 May 3, 2022
How to print the next 10 scheduled dates to validate my schedule is set up correctly?
prefect-2-0 ,  local-development ,  scheduling ,  async ,  cron ,  unit-testing ,  rrule
0 29 April 29, 2022
Does Prefect 2.0 support filters and adjustments e.g. to restrict the schedule to only run during business hours?
migration-guide ,  prefect-2-0 ,  local-development ,  scheduling ,  interval-schedule ,  cron ,  unit-testing ,  rrule
0 38 April 29, 2022
How to create a flow run from deployment (orchestrator pattern)?
prefect-2-0 ,  deployment ,  flow-of-flows ,  create-flow-run ,  orchestrator-pattern ,  create_flow_run_from_deployment
8 70 April 27, 2022
How can I trigger data quality checks as soon as my data arrives in a data lake?
prefect-1-0 ,  prefect-2-0 ,  event-driven ,  data-quality ,  data-validation
0 59 April 26, 2022
How to get finished futures as soon as they complete in a non-blocking way?
prefect-2-0 ,  async
1 40 April 24, 2022
How can I run tasks in parallel?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  dask ,  executor ,  task-runner ,  parallel-processing ,  dask-task-runner ,  infrastructure ,  multithreading ,  multiprocessing ,  basics
0 75 January 31, 2022
I have a flow run that got stuck in a Running state. How can I cancel it from the Orion client?
prefect-2-0 ,  states ,  cancellation ,  final-state
0 26 April 23, 2022
Can I set a custom name of a task or task run at run-time in Prefect 2.0?
prefect-2-0 ,  task-name ,  task-args ,  task-run-name
0 50 March 30, 2022
What is the setting PREFECT_ORION_DATABASE_CONNECTION_TIMEOUT used for and can this setting be used to mitigate my timeout errors?
prefect-2-0 ,  timeout
6 41 April 22, 2022
How to ensure that my flow run is considered failed when any of mapped tasks failed?
prefect-1-0 ,  reference-tasks ,  state-dependencies ,  mapping ,  unmapped
0 21 April 20, 2022
How to deserialize flow run results generated by default S3 storage?
prefect-2-0 ,  storage ,  aws ,  s3 ,  pickle ,  results ,  s3-storage
0 22 April 20, 2022
How to set upstream task dependencies between a ShellTask and downstream mapped tasks?
prefect-1-0 ,  state-dependencies ,  mapping ,  upstream_tasks ,  unmapped
0 27 April 20, 2022
How can I create a subflow and block until it’s completed?
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  flow-of-flows ,  create-flow-run ,  orchestrator-pattern ,  subflows ,  wait_for_flow_run ,  to-review ,  start_flow_run
0 186 January 28, 2022
How can we pause a task until a specified time and how to resume the same task before the timeout with some external triggers
task-library
4 37 April 14, 2022
How can I cancel several Submitted flow runs at once?
prefect-1-0 ,  graphql ,  cancellation ,  submitted-state
0 33 April 13, 2022