I am running 100 tasks in total and I expect them to be running in batches of 10, but I see a burst at the start of around 60 tasks being run in parallel then the limit starts being enforced and the remaining tasks do get processed 10 by 10.
If I change the concurrency limit of my tag to 5, I still get a burst of around 60 tasks at first then the remaining tasks are processed in batches of 5.
Is this initial burst behavior expected, is there a wait to configure it?
I ended up trying out the same code on my end and the task concurrency worked as expected. Some trouble shooting techniques that may be helpful are redeploying the code, and looking in the parameters of the task runs to ensure the tag is present.
I have the same problem. The concurrency limit was set to 3 but my tasks run with the same pattern as in Clement’s case. I’m running Prefect locally in docker-compose. Would it be helpful to create a repo with example project to debug this issue? @Matt_Conger