Unable to activate a virtual environment for execution of a job using Cloud as deployment

Help
, ,
1

Setup: I use a laptop to developing flows and tasks, which wil be deployed to my Prefect Cloud account. I use poetry to manage my virtual environments. I have an external server where, where the prefect worker is running in docker. For simplicity for this question, location of my flows is set to local.

Goal: I want each of my deployments to run in their designated virtual environment, i.e., using the pyproject.toml and poetry.

Problem: I am unable to use the virtual environment, even though I can use the utility step prefect.deployments.steps.run_shell_script to run poetry install and poetry shell, this does not trickle down to how prefect is executing the flows. Which I guess makes sense, as Prefect has probably already initiated the execution of the whole deployment pipeline.

Question: Is there a way, where I can define prefect.yaml file, which will deploy my code to Prefect Cloud, that when it is executed, will use my virtual environment as defined in pyproject.toml with poetry?

Thanks for a great product :slight_smile: I hope someone can with my issue.

Below is my prefect.yaml file, which hopefuly gives a picture of what I am trying to do:

# Welcome to your prefect.yaml file! You can you this file for storing and managing
# configuration for deploying your flows. We recommend committing this file to source
# control along with your flow code.

# Generic metadata about this project
name: pr_offer_check
prefect-version: 2.11.0

# build section allows you to manage and build docker images
build: null

# push section allows you to manage if and how this project is uploaded to remote locations
push: null

# pull section allows you to provide instructions for cloning this project in remote locations
pull:
  - prefect.deployments.steps.set_working_directory:
      directory: /mnt/Projects/pr_offer_check

  - prefect.deployments.steps.run_shell_script:
      id: poetry install
      script: poetry install
      stream_output: true
  - prefect.deployments.steps.run_shell_script:
      id: poetry shell
      script: poetry shell 
      stream_output: true

# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: pr offer check
  version: null
  tags: []
  description: null
  entrypoint: main.py:main
  parameters: {}
  work_pool:
    name: my-process-pool
    work_queue_name: null
    job_variables: {}
  schedule: null