I am running around 80 flows that all run every minute.
I am planning to run more flows, maybe in 100s and all of them running per minute. Am I using Prefect incorrectly and can it handle this scale?
I checked out this link from the docs - ECS Agent | Prefect Docs. I am already running the ECS agent with the specified options for retry mode but these errors still happen. Is there any way to fix this? I don’t really need Prefect to create new task definitions on every flow run, can I disable that behaviour? I am also using a custom task definition, could it be because of that?