Hi everyone,
I have a question regarding the functionality of prefect. I would like to use it for creating workflows for scientific computing (solving PDEs mainly and the related pre- and postprocessing).
I work mainly in python, so the task and workflow creation in prefect seem pretty nice.
However, I would like to have the following functionality: As my tasks involve heavy computations, I would like to be able to cache their results - which are files containing, e.g., the finite-element meshes, solutions of PDEs, and post processing images and videos.
As far as I could tell, prefect only caches the “return” value of the tasks - is there also the possibility of caching files which are written by the tasks and how could this be achieved?
Thanks a lot in advance,
Sebastian