View in #prefect-community on Slack
@Vamsi_Reddy: Hi everyone, we have been experimenting with the ECSRun and set it up successfully. I was trying to test if the autoscaling feature works so I created a script to submit 100 flow runs in a loop. for the most part most of the runs succeeded however a few of them failed with error: An error occurred (ThrottlingException) when calling the DeregisterTaskDefinition operation (reached max retries: 2): Rate exceeded
does anyone know why some of the runs failed?
@Anna_Geller: try this https://docs.prefect.io/orchestration/agents/ecs.html#throttling-errors-on-flow-submission
ECS Agent | Prefect Docs
@Vamsi_Reddy: Hi @Anna_Geller could you let us know where to add these variables. we tried adding them to our container definition and are still experiencing the same issue.
@Anna_Geller: I think those must be set on the agent, unfortunately, not on the flow run container or ECS Run… I know it’s painful because you need to redeploy the agent now 
@Vamsi_Reddy: no problem we will give it a try again.
@David_Wang: We are currently using an ECS agent, so do we add the variables in the environment underneath the containerDefinitions like so:
"containerDefinitions": [
{
"name": "$ECS_SERVICE_NAME",
"image": "prefecthq/prefect",
"essential": true,
"command": ["prefect","agent","ecs","start","--run-task-kwargs","<s3://xxx-test-bucket/david/ecs-config.yaml>"],
"environment": [
{
"name": "PREFECT__CLOUD__API_KEY",
"value": "xxx"
},
{
"name": "PREFECT__CLOUD__AGENT__LABELS",
"value": "['dev']"
},
{
"name": "PREFECT__CLOUD__AGENT__LEVEL",
"value": "INFO"
},
{
"name": "PREFECT__CLOUD__API",
"value": "<https://api.prefect.io>"
},
{
"name": "AWS_RETRY_MODE",
"value": "adaptive"
},
{
"name": "AWS_MAX_ATTEMPTS",
"value": 10
}
],