When I don't set an image in the ECS task definition, I'm getting: "An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Container.image should not be null or empty"

TL;DR: you have to set an image in your task definition, but you can then override it on your ECSRun run configuration.

View in #prefect-community on Slack

Scott_Aefsky @Scott_Aefsky: Hi folks. I’m trying to get my tasks to log to Cloudwatch through the ECSRun configuration. I’m testing out specifying a yaml config in task_definition_path, and running into an issue. My flows run on different ECR images, so I don’t want to specify the image in the Task Definition file. However, when I leave it out, and specify it in the constructor:

run_config = ECS_RUN(image=ECR_IMAGE, task_definition_path='s3://<path>'

I get an error that container.image should not be empty. My task definition file that I’d like to use is:

networkMode: awsvpc
cpu: 1024
memory: 2048
containerDefinitions:
  - name: flow
    logConfiguration:
      logDriver: awslogs
      options:
        awslogs-group: test_prefect_log_group
        awslogs-region: us-east-1
        awslogs-stream-prefix: ecs-prefect
        awslogs-create-group: "True"

Is there any way to use a Task Definition file without specifying an image? Or is there another/better way to make sure my ECS tasks get logged? Thanks for any help.

Kevin_Kho @Kevin_Kho: Even if it’s in the task_definition, it will get overriden by the RunConfig

Scott_Aefsky @Scott_Aefsky: Oh, interesting. So I can put a non-existent image name in the task_definition to get past the error, and the RunConfig will override it?

Kevin_Kho @Kevin_Kho: I believe so because 3 things are merged before the ECS task is registered. The agent task_definition , the run config task definition, and the run configuration like cpu and memory. The priority is in that order also, so the RunConfig should override. Still though, I am not seeing why you get this error because the default task definition has none, but I guess we must be filling it.

Scott_Aefsky @Scott_Aefsky: Yep, it worked like I wanted with a non-existent image
The full error when I tried to leave the image name out of the task_definition file was:

An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Container.image should not be null or empty.

Kevin_Kho @Kevin_Kho: Ah ok. Does that work for you then?

Scott_Aefsky @Scott_Aefsky: Yep, I am all set

Kevin_Kho @Kevin_Kho: Nice!

Scott_Aefsky @Scott_Aefsky: Thanks as always for the quick help, Kevin!