By default, when i run a deployment from the UI (orion server) and start an agent, the command line “python -m prefect.engine” is executed in a directory at “/tmp/tmp*prefect”. However, I am working on a server where the “/tmp” directory is very limited in both space and access time. Can I change the current directory of the execution to another one on my computer (like /home/tmp*prefect) ?
I use the infrastructure Process to run my flow.
Could you elaborate? You don’t need to store anything there, it’s only for the duration of a flow run
Digging more, I found out that you might be able to change that by setting one of those variables (I’m not sure which one, though):
prefect config set PREFECT_LOCAL_STORAGE_PATH="/your/custom/path"
prefect config set PREFECT_HOME="/your/custom/path"
I changed both variables and it didn’t work for me. I run a flow with just “print(os.getcwd())” inside and it still print me “/tmp/tmp3k1tb_3hprefect”
I work on a server where /tmp is not on the same disk than /home , the size is very limited (about 100 Mo)
gotcha, I can look at this in 1-2 weeks again
It is very important problem, for me too, I create Flows for processing big files (~20-30GB) and prefect use tmp dir for store it (I noticed it when Flows was running), prefect cleans after flow executed it is good but I’d like to set the dir for temp files cause boot memory is 50GB only and the boot memory is expensive to use and the better way is to use extra-disk or may be store every data at .prefect/ (which location we already can control)
Now I see just mount extra space to tmp dir
That’s a neat idea and definitely viable. Check also prefect settings, there’s a settings variable you can use to adjust which directory is used for that by Prefect