@ale: Another question, this time related to Prefect 2.0 in general: looking at some of the available collections maintained by Prefect, I’ve noticed that the async pattern is frequently used.
Are there any suggestions/best practices/recommendations regarding the use of the async pattern we should take into account when developing a new collection?
@Anna_Geller: We were talking about this internally and here was an answer from Michael: “You shouldn’t need to use async unless you want async. The async collections tasks can be treated as normal sync tasks.”
Generally, Michael has implemented a lot of magic, allowing you to combine sync and async tasks in a flow.
Regarding building collections, async seems to be preferable since you can then still call those as normal sync tasks. But @alex can correct me if I’m wrong and explain it better
@alex: Yep, what Anna said is 100% correct. We default to async tasks so that if users want to use async flow the tasks will be no blocking, but because of magic those tasks can still be used in sync flows. It’s the best of both worlds!
By default, synchronous tasks are now run in a thread to avoid blocking the event loop. For this reason, I think it should be fine to write sync or async tasks in a collection depending on the interface in the library you are wrapping.