I am sure this has been asked before but I couldn’t find it anywhere… Is there a limit (or rule of thumb) to how much data you can pass between prefect tasks?
I have always thought you weren’t supposed to pass large amounts of data (e.g. dataframes) between tasks, which I am assuming I read somewhere in the docs at one point or another. I was trying to find where I read this and couldn’t find it again. We are using prefect 1.0; not sure if this has changed in prefect 2.0.