Scheduled Flow (every 30min) crashed after running on local server for 2 days

Context :
I am trying to run a scheduled Prefect flow (every 20min) on a local server. However, after running the flow for two days (with only Completed states), it crashes.

Error :

I’ve tried digging but I can’t figure out and solve the error:
For me it’s a memory problem related to ~/.prefect/prefect.db

What I tested :

  • Reduce the number of logs to avoid overloading the database (maybe it’s not enough)
  • Set concurrency limit on some tasks

Thank you in advance for your help

Hello there! Thanks for making this post.
From what I gather, it seems like the Prefect server that you’re self hosting is crashing/becoming unavailable after a certain period of time. In this case, it most likely went offline after two days. In order for flow runs to be run against Prefect Server, the server needs to be available at all times. For a more robust setup, I’d recommend checking out our helm chart example or this article for self-hosting Prefect with Docker Compose. Self hosting for production can be engineering intensive. If you don’t want to host the orchestration API yourself, I’d recommend checking out Cloud 2.0. We have a very generous free tier that would allow you to invite two collaborators and still execute your flows where the worker is started in your infrastructure.