When using ECSRun I'm getting an error: "An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Invalid setting for container 'flow'. At least one of 'memory' or 'memoryReservation' must be specified"

View in #prefect-community on Slack

Rhys_Mansal @Rhys_Mansal: Sorry to ask multiple questions but I have a flow that is running on ECR which fails with the following message

An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Invalid setting for container 'flow'. At least one of 'memory' or 'memoryReservation' must be specified. 

I have set memory (and cpu) in the ECSRun task_definition (both quoted and unquoted) and still get this error. Does anyone know what I might try?

Kevin_Kho @Kevin_Kho: No worries! Here to help. Can I see your task definition?

Anna_Geller @Anna_Geller: ECS is super specific about what you need to set there. We saw a similar issue when the image was not set explicitly - you need to check the required arguments for an ECS task definition and you need to set them all explicitly unless you let Prefect create and register a task definition for you - if you need some examples, check out those here with the name “ecs” in it

Prefect Community: When I don’t set an image in the ECS task definition, I’m getting: “An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Container.image should not be null or empty”

GitHub: packaging-prefect-flows/flows at master · anna-geller/packaging-prefect-flows

Rhys_Mansal @Rhys_Mansal: thank you. I think it’s the lack of the explicit image then.

Kevin_Kho @Kevin_Kho: no no. yours says memory. I think it was some memory also specified in the task definition

Rhys_Mansal @Rhys_Mansal: currently I have this

with Flow("flow-name",
          state_handlers=[run_name_ts],
          storage=Module("scheduling.flows"),
          run_config=ECSRun(
              task_definition={
                  'containerDefinitions': [
                      {
                          'name': 'flow',
                          'image': '<http://some-string.amazonaws.com/container:0.1.0|some-string.amazonaws.com/container:0.1.0>'
                      }
                  ]
              },
              memory=512,
              cpu=256,
              labels=["dev", "other-string"])) as flow:

Solution

Kevin_Kho @Kevin_Kho: It wants a memory value inside containerDefnitions
But with just setup, so need to give task definition, just give the image to ECSRun and you should be good. Prefect makes that flow container by default anyway

Rhys_Mansal @Rhys_Mansal: excellent. Thank you