How to pass custom worker resource annotations to Dask scheduler when using DaskExecutor?

View in #prefect-community on Slack

Andreas_Ntonas @Andreas_Ntonas: Hi Everyone!
I have a small question regarding dedicated execution of prefect tasks using a DaskExecutor. For example let’s say I have some special tasks in a flow that need to be executed specifically on a dedicated machine. Is it possible to provide this information in Prefect?

Anna_Geller @Anna_Geller: Could you explain the problem you try to solve? Otherwise, I’ll give you answers that will only confuse you further.

Kevin_Kho @Kevin_Kho: Yes I present to you Prefect’s least used feature!! Look at the DaskExecutor docs, you can use worker resources annotation to point a task to a specific worker. For example, you have a worker with a GPU

Andreas_Ntonas @Andreas_Ntonas: Hahah, Thank you! This solved the problem. I was experimenting with a much more difficult workaround of creating flows with a different executors on the dedicated machines for these tasks and then creating an orchestrating flow, but the Worker Resources are much simpler. Thank you!!:pray: