How do i get Prefect agents on a different machine from the prefect server host? I am trying to create scalability options
Just download your repo, set your env and spam a agent. Just remember to set the host and port to the same address the server is.
Server port must be open for this to work.
You can check whether the agent is active or not in the ui.
But the agent will just crash if there is no server to attach. So server must be up before setting agent
I am working with following approach:
Run server on one machine. Run worker(s) on other machines.
In worker machine, set PREFECT_API_URL environment variable that communicates with the prefect server. The workers now should be able to poll for new flow runs that are submitted to the server. You should note that, the worker and server are communicating with HTTP so server needs to be able http requests.
For now, I only have one worker machine and one worker. With this approach you should be able to extend it to multiple workers on different worker machines.