I have been trying to set a concurrency limitation using a tag on a task like this:
@task(tags=["noDOS"])
def print_values():
time.sleep(3)
@flow()
def my_flow():
for i in range(100):
print_values.submit()
my_flow()
My tag concurrency limit is set to 10 using the command:
prefect concurrency-limit create noDOS 10
When I run it (by calling python directly to run the script above) I get this execution pattern in the GUI (what I see in the terminal matches with the web interface):
I am running 100 tasks in total and I expect them to be running in batches of 10, but I see a burst at the start of around 60 tasks being run in parallel then the limit starts being enforced and the remaining tasks do get processed 10 by 10.
If I change the concurrency limit of my tag to 5, I still get a burst of around 60 tasks at first then the remaining tasks are processed in batches of 5.
Is this initial burst behavior expected, is there a wait to configure it?
I ended up trying out the same code on my end and the task concurrency worked as expected. Some trouble shooting techniques that may be helpful are redeploying the code, and looking in the parameters of the task runs to ensure the tag is present.
Thanks a lot for having a look into this issue. You raise a good point by talking about deployment, in my testing I was running the flow by calling python on it directly. So I tried a few more stuffs:
My first message was running on Ubuntu 22.4, so I tried it also on windows 11 but I get the same issue
Using a deployment to package my flow, but this didn’t help either, here is a screenshot of one of the tasks where the tag is properly set
Even weirder, by using a deployment I had to start a work pool which is itself limited to 10 concurrent tasks. But neither the work pool limit nor the tag on the task prevented this early burst.
I have the same problem. The concurrency limit was set to 3 but my tasks run with the same pattern as in Clement’s case. I’m running Prefect locally in docker-compose. Would it be helpful to create a repo with example project to debug this issue? @Matt_Conger
