ECS task role set on ECS agent startup command isn't applied to flow's run configuration

Archive
, , ,
3

Thanks, I’ll look at those links. I think since the task role is defined in ECSRun it should be put through the task definition, but doesn’t seem to be. Here is my setup:

Prefect Diagnostics Output

{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "macOS-12.2.1-x86_64-i386-64bit",
    "prefect_backend": "cloud",
    "prefect_version": "1.1.0",
    "python_version": "3.10.2"
  }
}

Flow config

STORAGE_CONFIG = GitHub(
            repo="XXXX/workflows",                            # name of repo
            path="workflows/marketing/test/test/flow.py",                    # location of flow file in repo
            access_token_secret="personal access token")

RUN_CONFIG = ECSRun(labels=['test'],
        image="XXXX.dkr.ecr.us-west-2.amazonaws.com/workflows/test:latest",
        task_role_arn='arn:aws:iam::XXXX:role/prefect-test-task-role'
        )
with Flow("Test", storage=STORAGE_CONFIG, run_config=RUN_CONFIG) as flow:
    ...

if __name__ == "__main__":
    flow.register(project_name="marketing")

Flow Registration

I had issues registering but following this issue was able to get it to work.

python -m workflows.marketing.test.test.flow

ECS Agent task definition

This is the ECS task definition for the container running the Agent

{
    "taskDefinitionArn": "arn:aws:ecs:us-west-2:XXXX:task-definition/prefect-test:4",
    "containerDefinitions": [
        {
            "name": "prefect",
            "image": "prefecthq/prefect:latest-python3.8",
            "cpu": 0,
            "links": [],
            "portMappings": [],
            "essential": true,
            "entryPoint": [],
            "command": [
                "prefect",
                "agent",
                "ecs",
                "start",
                "--execution-role-arn",
                "arn:aws:iam::XXXX:role/prefect-test-service-role",
                "--task-role-arn",
                "arn:aws:iam::XXXX:role/prefect-test-task-role",
                "--cluster",
                "prefect-test"
            ],
            "environment": [
                {
                    "name": "PREFECT__CLOUD__AGENT__LABELS",
                    "value": "['test']"
                },
                {
                    "name": "PREFECT__CLOUD__API",
                    "value": "https://api.prefect.io"
                },
                {
                    "name": "PREFECT__CLOUD__AGENT__LEVEL",
                    "value": "INFO"
                }
            ],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "secrets": [
                {
                    "name": "PREFECT__CLOUD__API_KEY",
                    "valueFrom": "arn:aws:ssm:us-west-2:XXXX:parameter/prefect/ecs_agent/api_key"
                }
            ],
            "dnsServers": [],
            "dnsSearchDomains": [],
            "extraHosts": [],
            "dockerSecurityOptions": [],
            "dockerLabels": {},
            "ulimits": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "prefect-test",
                    "awslogs-region": "us-west-2",
                    "awslogs-stream-prefix": "prefect-test"
                },
                "secretOptions": []
            },
            "systemControls": []
        }
    ],
    "family": "prefect-test",
    "taskRoleArn": "arn:aws:iam::XXXX:role/prefect-test-task-role",
    "executionRoleArn": "arn:aws:iam::XXXX:role/prefect-test-service-role",
    "networkMode": "awsvpc",
    "revision": 4,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
        },
        {
            "name": "ecs.capability.execution-role-awslogs"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.17"
        },
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role"
        },
        {
            "name": "ecs.capability.secrets.ssm.environment-variables"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        },
        {
            "name": "ecs.capability.task-eni"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EC2",
        "FARGATE"
    ],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "512",
    "memory": "1024",
    "registeredAt": "2022-04-19T02:06:54.479Z",
    "registeredBy": "XXXX",
    "tags": []
}

Flow task definition

When a new flow run is sent the following task definitions get automatically created. It contains the execution role but not the task role.

{
    "taskDefinitionArn": "arn:aws:ecs:us-west-2:XXXX:task-definition/prefect-test-dde22515-bf71-4f9a-bf6e-b4bbe23cf06d:1",
    "containerDefinitions": [
        {
            "name": "flow",
            "image": "XXXX.dkr.ecr.us-west-2.amazonaws.com/workflows/test:latest",
            "cpu": 0,
            "portMappings": [],
            "essential": true,
            "environment": [
                {
                    "name": "PREFECT__CONTEXT__IMAGE",
                    "value": "XXXX.dkr.ecr.us-west-2.amazonaws.com/workflows/test:latest"
                }
            ],
            "mountPoints": [],
            "volumesFrom": []
        }
    ],
    "family": "prefect-test-dde22515-bf71-4f9a-bf6e-b4bbe23cf06d",
    "executionRoleArn": "arn:aws:iam::XXXX:role/prefect-test-service-role",
    "networkMode": "awsvpc",
    "revision": 1,
    "volumes": [],
    "status": "INACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.ecr-auth"
        },
        {
            "name": "ecs.capability.execution-role-ecr-pull"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        },
        {
            "name": "ecs.capability.task-eni"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EC2",
        "FARGATE"
    ],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "1024",
    "memory": "2048",
    "registeredAt": "2022-04-19T23:22:00.584Z",
    "deregisteredAt": "2022-04-19T23:22:01.651Z",
    "registeredBy": "arn:aws:sts::XXXX:assumed-role/prefect-test-task-role/46032a0d17944ebf94c5003385879843",
    "tags": []
}

Prefect Task Role

There are two policies associated with the role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcs",
                "ec2:DeleteSecurityGroup",
                "ecs:CreateCluster",
                "ecs:DeleteCluster",
                "ecs:DeregisterTaskDefinition",
                "ecs:DescribeClusters",
                "ecs:DescribeTaskDefinition",
                "ecs:DescribeTasks",
                "ecs:ListAccountSettings",
                "ecs:ListClusters",
                "ecs:ListTaskDefinitions",
                "ecs:RegisterTaskDefinition",
                "ecs:RunTask",
                "ecs:StopTask",
                "iam:PassRole",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogGroups",
                "logs:GetLogEvents"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ssm:DescribeParameters",
                "ssm:GetParameters",
                "ssm:GetParametersByPath",
                "ssm:GetParameter"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

Prefect Task Execution Role

Contains two policies listed below

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": "ssm:DescribeParameters",
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "ssm:GetParameters",
                "ssm:GetParametersByPath",
                "ssm:GetParameter"
            ],
            "Resource": "arn:aws:ssm:us-west-2:XXXX:parameter/prefect/*",
            "Effect": "Allow"
        }
    ]
}