Checkpoint/restart capability

Archive
1

Hi, I am evaluating Prefect. I will firstly say, thank you for answering all of the questions I’ve posted thus far.

What I’d like to know is if there are any plans on the roadmap (or if there is an existing capability that I may have missed) to have an ability to resume a flow that has failed. What I see is that I can retry tasks that fail n times. What I am looking for is the ability to find a flow run that has failed, and start executing from the task that failed. The scenario would be a long running process with a few tasks where I would not want to re-execute the tasks that succeeded, I would skip those, and start execution from the task that failed.
The way I see it now would be that since the code that orchestrates the tasks is my own code (ie calling one task after another task), the onus would be on me to somehow keep track of tasks that have run and then if the deployed flow is re-executed, to somehow retrieve that info and skip over the tasks that do not need to be run again.

Thanks.

1 Like
2

This is not directly possible currently but you can easily restart the flow run in any way you want locally, even by commenting out the parts that you don’t want to run again

You can think of Prefect as a much more dynamic coordination plane that allows you to run your tasks and flows in a way you design it so you have full control over how you would implement that. The easiest is by rerunning manually only the parts you need. Another option is the orchestrator pattern: