EKS vs ECS for Prefect 2.0

Archive
, ,
1

Hi, I’ve been running Prefect 1.0 using agents running on AWS ECS Fargate.

I am looking to migrate to Prefect 2.0, and am wondering, what are some differences between using ECS vs EKS for deployment? I found this doc that outlines how to set up the prefect agent with EKS (Distributed data pipelines made easy with AWS EKS and Prefect | by Anna Geller | Towards Data Science), but I wanted some recommendations to see if using EKS or ECS is better for certain workflows. For example, is EKS or ECS better or worse for tasks that are compute intensive? Is there an official recommendation from the prefect community to use EKS or ECS with Prefect 2.0?

Thank you!

1 Like
2

it depends a lot on your use case. For ECS, we started building out a recipe with automated agent deployment and CI/CD here

3

Thanks @anna_geller for the response. I am trying to deploy the prefect agent to EKS following your instructions in Distributed data pipelines made easy with AWS EKS and Prefect | by Anna Geller | Towards Data Science.

I want to better understand what this command is doing:

prefect agent install kubernetes -t <MyEKS_on_Fargate_K8s_Token> \
    --rbac | kubectl apply -f -

Is this the prefect agent helm chart?

Is there an official agent and server helm chart somewhere, and if so, could you please tell me where?
Thanks!

4

don’t follow instructions from this post - this post is ultra outdated

check out the resources listed for 2.0 and kubernetes as shown here Docs - Prefect Community

for Helm, see

1 Like
5

thank you! we were able to successfully deploy the agent to our EKS cluster and run a hello world flow

2 Likes