Just a tiny business / hobbyist here - where the Prefect cloud pricing doesn’t make any sense.
I have to use the Prefect server either on my local machine, but some concerns: if my local machine shuts down and restarts, does the Prefect server still preserve the job states and can pick up from where it left off?
Also - what’s it like to self-host the Prefect server on EC2? Is it fairly straight forward? Any links to resources would be greatly appreciated. Thanks!
Hello,
Based on my small knowledge and if I understood correctly, you can host a prefect server on you machine (or on an instance on cloud), create a postgre SQL database and link it to prefect server (documentation is very clear here), thus you’ll keep all the information about flows and executions even if the server shuts downs (I don’t know if there is a way to pick up a job where it left off).
Running jobs on EC2 might be challenging and the documentation about it is limited specially when it comes to error handling.
Hope that helps.