This post demonstrates how to set up a systemd service to run a Prefect 2 agent.
A systemd service is an ideal way to run a Prefect agent on a Linux VM or physical server if you want to:
- Automatically start a Prefect agent when Linux starts.
- Automatically restart a Prefect agent if it crashes.
Here’s what you’ll need before proceeding:
- A working Linux machine with systemd and Python 3.7 or later.
- A superuser account, i.e. you can run
- A Prefect Cloud account, or a Prefect Orion server running somewhere on your local network.
- A Prefect deployment with a work queue your agent can connect to.
First, create a user account for the Prefect agent. While you can run the agent as root, it’s best to avoid doing so unless you are sure you need to.
In a terminal, run:
sudo useradd -m prefect sudo passwd prefect
When prompted, enter a password for the
Next, log in to the
prefect account by running:
sudo su prefect
Now, it’s time to install Prefect. Run:
pip install prefect
This assumes you are installing Prefect globally, not in a virtual environment. Running a systemd service from a venv will work, but when you set up the service below, you’ll need to change the ExecPath to target the
prefect application in the
bin subdirectory of your virtual environment.
Next, you must set up Prefect so it will know which server to connect to.
If you are connecting to a local Prefect Orion server, run the following and substiture the IP address of your server:
prefect config set PREFECT_API_URL=http://your-orion-server-IP:4200
If you are connecting to Prefect Cloud, follow the instructions to obtain an API key and then run:
prefect cloud login -k YOUR_API_KEY
When prompted, choose the Prefect workspace you’d like to log in to.
Finally, run the
exit command to sign out of the
prefect Linux account. This switches you back to your sudo-enabled account so you will be able to run the commands in the next section.
Now you’re ready to set up your Prefect agent systemd service. Run the following commands:
cd /etc/systemd/system sudo vim prefect-agent.service
Feel free to use another editor like Nano or Emacs instead of Vim if you prefer. Add the following to the file:
[Unit] Description=Prefect Agent [Service] User=prefect WorkingDirectory=/home/prefect ExecStart=/usr/local/bin/prefect agent start -q YOUR_WORK_QUEUE_NAME Restart=always [Install] WantedBy=multi-user.target
Make sure you substitute your own work queue name. When you’re done, save the file and exit. If you’re stuck in Vim, don’t worry! Just hit the escape key, type
:wq!, then press the return key.
systemctl daemon-reload to make systemd aware of your new service.
systemctl enable prefect-agent to enable the service. This will ensure it runs when your system boots.
systemctl start prefect-agent to start the service. And that’s it! You now have a systemd service that starts a Prefect agent when your system boots, and will restart the agent if it ever crashes.