New to all this. How would I set up 50 python scripts for monitoring?

Help
1

I have done through the tutorial but something just hasn’t clicked yet. Let’s say I currently have 50 different python scripts handling ETL jobs. How would I use prefect to schedule/monitor them?

From the tutorial, each python script that has a deployment (.serve) needs to be running, but it clearly doesn’t make sense to open 50 terminals and have each of them running a .py file. I also don’t want to combine all the 50 files into one giant file.

Forgive me, I am not a python developer (I have intermediate knowledge) , but was recently put in charge of automating these ETL processes and was hoping Prefect would be a good way to go.

2

I’m not an expert on Prefect, just started using because I had the same problem as you described. If you want to run a single terminal, you need to combine all those files into one giant file with different flows and deployments, yes. Then I think the next step would be separating them into packages if it suits your needs.