FileNotFoundError: [Errno 2] No such file or directory creating a flow run from a deployment

Help
1

My deployment is as follows:

from prefect.deployments import Deployment
from prefect.filesystems import GitHub
from fl_blob_ingestion import main_flow


storage = GitHub.load("athenix-prefect")

deployment = Deployment.build_from_flow(
    flow = main_flow, # name of the flow function
    name = "dp_blob_ingestion_zonda", # name of deployment
    work_queue_name = "blob_ingestion_zonda",
    path="blob_ingestion",
    storage = storage
    )

if __name__ == "__main__":
    deployment.apply()

I get this error everytime I run it

FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\galtman\\AppData\\Local\\Temp\\tmp0t1_en1hprefect\\fl_blob_ingestion.py'

not sure what the problem is, it worked before when I made a deployment the same way about a month ago.

2

can you try deployment from CLI? here is a list of resources you can check:

3

I’ve tried deploying every way possible, I still get the same error. I’ve done both on Orion and the Cloud, as well as using the CLI and using the python object for each.

It worked a month ago so not sure what may have changed.

4

image
image1131×399 43.5 KB 

FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\galtman\\Documents\\pref_temp\\fl_blob_ingestion.py'

.
.

how can I get it to look at this path instead?
'C:\\Users\\galtman\\Documents\\pref_temp\\blob_ingestion\\fl_blob_ingestion.py'

my path is showing in the error message now since I’ve added it here in the process infrastructure block for the deployment
image

So now it builds the whole thing in that path but I have my main flow file in

..\\pref_temp\\blob_ingestion\\fl_blob_ingestion.py

So I need that earlier method to point to that path instead. Any idea how it can be done? @anna_geller

5

you could leverage --path argument on the prefect deployment build CLI, or use Process block instead at first, rather than the GitHub block

6

Can you give me an example of what the path argument does? I’ve also tried putting it in the path argument in the CLI but I get the same result. I’ve tried all these in the yaml

path: 'blob_ingestion'

path: 'blob_ingestion\'

path: '\blob_ingestion\'
7

I meant setting this on the CLI argument, not in YAML

you could read more in the CLI docs or by adding --help in the CLI on any command

8

Isn’t it the same thing though? Setting it through CLI or adding it in the yaml deployment file? Either way, I did that but still get the same result.

Here is my cli execution:

prefect deployment build fl_blob_ingestion.py:main_flow -n 'dp_blob_ingestion_zonda' -q 'default'  -sb github/github-test --path 'blob_ingestion' --apply
9

I have found the solution thanks to @EmilRex

I had to run the deployment from the root of the directory instead of at a deeper level.

For CLI:
Didn’t work

(venv) PS C:\Users\galtman\Documents\proj_folder\ingestion> prefect deployment build fl_ingestion.py:main_flow 
-n 'ingestion_zonda' 
-q 'ingestion_zonda' 
-sb github/my_repo
--path ingestion
--apply

Worked

(venv) PS C:\Users\galtman\Documents\proj_folder> prefect deployment build ingestion/fl_ingestion.py:main_flow 
-n 'ingestion_zonda' 
-q 'ingestion_zonda' 
-sb github/my_repo
--apply
1 Like