Contribution from Bernard Greyling:
I’ve been playing around with the idea using prefect in conjunction with JuiceFS for a distributed runtime environment.
In essence the idea is to use JuiceFS to mount a distributed files system to K8s pods to act as a shared virtual filesystem.
To simplify local development one simply has to manage a the runtime environment variables appropriately.
In this example repo I attempt to illustrate a basic local configuration.
Any ideas and comments would be much appreciated!"