I am currently trying to run a simple deployment inside of a docker container in a similar fashion to the prefect docker tutorial. I am trying to run my deployments and build my containers locally on my linux machine.
My build/pull configuration are specified as the following:
# build section allows you to manage and build docker images build: - prefect_docker.deployments.steps.build_docker_image: id: build_image requires: prefect-docker>=0.3.0 image_name: docker-tutorial-image tag: latest dockerfile: Dockerfile # push section allows you to manage if and how this project is uploaded to remote locations push: # pull section allows you to provide instructions for cloning this project in remote locations pull: - prefect.deployments.steps.run_shell_script: id: viewdir script: ls stream_output: true - prefect.deployments.steps.set_working_directory: directory: /opt/prefect/
I added the ls command with my pull step so I can see what my working directory looks like when I’m trying to run the flow. This is the output I get during the pull phase:
> Running run_shell_script step... dist entrypoint.sh requirements.txt /usr/local/lib/python3.8/runpy.py:127: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) 15:08:30.945 | ERROR | Flow run 'heavenly-carp' - Flow could not be retrieved from deployment. Traceback (most recent call last): File "<frozen importlib._bootstrap_external>", line 839, in exec_module File "<frozen importlib._bootstrap_external>", line 975, in get_code File "<frozen importlib._bootstrap_external>", line 1032, in get_data FileNotFoundError: [Errno 2] No such file or directory: 'flows/app.py' The above exception was the direct cause of the following exception:
As you can see above, for some reason the directory called “flows” which has the python file with my flow is not found in the docker container.
My docker file looks like the following:
FROM prefecthq/prefect:2.11.0-python3.8 COPY requirements.txt . RUN pip install -r requirements.txt --trusted-host pypi.python.org --no-cache-dir COPY flows /opt/prefect/flows RUN ls RUN pwd
What’s even more confusing to me is that during the build phase, I added RUN ls and RUN pwd, and you can clearly see that my “flows” directory got copied over properly inside the container. I am completely lost on what is happening. This is the output during the build phase:
Step 3/5 : RUN ls ---> Running in e658dbf088e8 dist entrypoint.sh flows requirements.txt ---> b93423469ac6 Step 4/5 : RUN pwd ---> Running in a46012a5cd2b /opt/prefect ---> fd460aca2b07