Is there a way to run tasks downstream from a specific point in a flow? For example, suppose a user has the following flow a >> b >> c >> d. After the flow has run, what is the best way to run tasks b to d?
To generalize, after all tasks in a flow have run, we may need to re-run a task that’s in the middle of the flow without running the upstream tasks. How do we ensure that the targeted task and all of its dependent downstream tasks are re-run? Thank you!
Hi rvjocassee - are you asking if you can start a flow run from any arbitrary point in the flow, only executing what you want?
You can use wait_for to designate upstream tasks that are dependencies.
Additionally, there are cache and results that can be used to not retry computationally intensive tasks.
Regarding your specific question, to retry just a part of a flow, you can use automations to trigger / re-trigger another flow run, but the burden would be within your design of what gets re-run - what would the expected behavior be from Prefect if a flow run is retried in this manner?