How to run flows from multiple poetry/pyenv virtual envs?

Help
1

Hello,

I work on a monorepo which includes several independent projects. Each project uses poetry/pyenv and has its own virtual environment. The monorepo’s structure looks like this:

monorepo/
  ..
  projects/
    project1/
    project2/

I have a prefect agent which is started by systemd and that is running fine in another virtual environment.

I’m able to run flows locally like this and they appear in Prefect Cloud UI:

> poetry run python projects/project1/flow/flow.py

However the deployments don’t work. I believe that the agent is not able to locate the project’s interpreter (ie, its virtual env) that is required to run the flow. The log message says ModuleNotFoundError: No module named [..].

Can someone explain what I need to do to get the agent to point the runner to the correct virtual env?

Sorry in advance if the question has already been asked or answered somewhere else, I’m new to prefect and haven’t found the information. Thank you.

2

Hi,

From what it look like in your message, the virtual machine you are running does not have the necessary dependencies to run the flows. Unless otherwise specified, your flow will run as a subprocess in the same VM as your agent. You can install those dependencies in your VM, or I would recommend checking out Docker.

The following links may be helpful.

Happy Engineering!