Pytesting Prefect tasks in a docker container

Help
1

In my CICD I want to include a step where I run unit and integration tests with Pytest before deploying.

The Dockerfile is based on a Prefect 2 image and sets up dependencies with Poetry:

FROM prefecthq/prefect:2-python3.9

RUN pip install -U pip \
    && curl -sSL https://install.python-poetry.org | python - 
ENV PATH="${PATH}:/root/.poetry/bin"

COPY . .
RUN pip install poetry 
RUN poetry config virtualenvs.create false
RUN poetry install

When I run Pytest on this image with

docker run $IMAGE /bin/bash -c "pytest"

I get the following error:

from prefect import flow, get_run_logger, task
E   ImportError: cannot import name 'flow' from 'prefect' (/opt/prefect/__init__.py)

It appears there might be a conflict in how and what it’s attempting to import with from prefect import. Furthermore, when I try to use disable_run_logger() from prefect.logging, it give a ModuleNotFoundError for prefect.logging, while a manual inspection finds the relevant files.

Does anyone have an idea? Thanks!

2

you can try using a different working directory than /opt/prefect e.g. WORKDIR /app/

I’ve never seen this issue when using just pip - any reason why you use poetry in Docker? typically you would use either docker or poetry, not both

3

Setting the workdir worked, thanks a lot!

No specific reason to use Poetry on the deployment besides convenience since Poetry is used to manage the dependencies of the project. Any suggested setups you can point to?

1 Like