How to run your Prefect agent as a background process with supervisord?

Make sure that you are connected to the right profile.

View in #best-practices-coordination-plane on Slack

Jimmy_Le @Jimmy_Le: Here is a sample supervisor configuration file for running your agents in the background and making sure they restart in the event of a failure.

I’ve also added some venv environments so you can have different virtual environments for different work queues.

Add this to your /etc/supervisor/conf.d/ folder.

I’ve named mine prefect.conf:

[program:prefect]
command=/home/usr/venv/bin/python /home/usr/venv/bin/prefect agent start --work-queue "queue-name"
autorestart=true
autostart=true
stderr_logfile=/var/log/prefect.err.log
stdout_logfile=/var/log/prefect.out.log
environment=PATH="/home/usr/venv/bin",PREFECT_API_URL="",PREFECT_API_KEY=""

I’ve also set up a daemon service such that supervisor will start up and restart the agents in the event of a server outage or restart. Lmk if anyone is interested in this and I can share.

Jeff_Hale @Jeff_Hale: Thanks @Jimmy_Le! Those might be great to put in the discourse dataops section.

Prefect Community

1 Like