Hi All,
Thank you for being that generous with your time and work.
I’m migrating from prefect 2 to prefect 3 and I need to switch to the new deployment mechanism. Unfortunately it doesn’t work as I expect.
The old deployment was:
Deployment.build_from_flow(
name="...",
flow=my_flow,
parameters={..},
storage=RemoteFileSystem.load('minio'),
path='...',
version=os.getenv("GIT_COMMIT_SHA"),
infrastructure=DockerContainer(
auto_remove=True,
image='prefect-default-worker:2.20.9',
image_pull_policy='NEVER',
network_mode='bridge',
),
).apply()
Where ‘minio’ is a Remote File System block containing Minio connection data, prefect-default-worker:2.20.9 is an image based on prefecthq/prefect:2.20.9-python3.10 with all the flow’s dependencies installed. I have another container prefect-agent:2.20.9 with entrypoint: ["prefect", "agent", "start", "-q", "default"]
Reading the migration guide I get that I need to 1) Replace ‘infrastructure’ with ‘work_pool_name’ and 2) Replace ‘storage’ with a call to ‘flow.from_source’ 3) replace entrypoint: ["prefect", "agent", "start", "-q", "default"]
with entrypoint: ["prefect", "worker", "start", "-p", "default"]
The new deployment script looks like this:
my_flow.from_source(
source=RemoteFileSystem.load("minio"),
entrypoint='path to the file with @flow definition'
).deploy(
name="same",
work_pool_name='default',
parameters={same},
version=os.getenv("GIT_COMMIT_SHA"),
build=False,
push=False,
job_variables=dict(pull_policy="NEVER", network_mode="bridge", auto_remove=True)
)
Before the migration, the script uploads the flow’s python file (and the whole folder) to minio and then create/update the deployment.
Now it doesn’t seem to do the upload to Minio. The script throws an error: FileNotFoundError: [Errno 2] No such file or directory: ‘/var/folders/l9/q51p6spx7sb6j3gqdq5q7jhr0000gn/T/tmpi8gx9s0t/remote-file-system-minio/…’
If I manually put the files in Minio manually It seems to load the flow. I can see that a ScriptError is raised loading the flow’s python file. I’ll share the error after I clarify this issue.
So I wonder if I can keep the previous behaviour, or I should upload the flow’s definition to Minio manually.