Hi all. I am in the process of migrating an ETL app from Prefect 1.0 to Prefct 2.0. I’m running
prefect==2.7.1 locally. I’ve made an initial migration and it appears to be working fine, except there is one behaviour I’d like to change. My ETL consists of a number of tasks, all of them independent from each other (yet they run sequentially, per default settings, which is ok by me).
The desired behavior is that if one task fails, all subsequent tasks continue, without halting the
flow; this happened by default in Prefect 1.0 without me touching anything, but it’s not happening by default in Prefect 2.0. That is, whenever one of my tasks raises an exception, the entire flow stops. How can I replicate the behavior I had in Prefect 1.0? (which for my use case suited me better).
from prefect import flow, task @task def fail_fast(): raise Exception("Error raised for testing") @task def ask_question(): print("What is your favorite number?") @flow def run_entry_point(): fail_fast() # fail_fast().submit() # .submit() or not makes no difference ask_question() return 42 if __name__ == "__main__": print(run_entry_point())
I’ve looked into this thread: How can I handle errors, allowing my flow runs to continue until complete and not raise upon failure? and quite a few others, but I just can’t manage to find what “knob” to tweak to make it happen.
Thanks in advance.