View in #prefect-community on Slack
@Matt_Delacour: hi everyone 
We are interesting in using ECS Agent with Prefect 2.0. Is this something already available ?
Cannot find anything related to that in the Orion doc so far.
cc @Julian_Mehnle
@Kevin_Kho: I don’t think that has been implemented yet. Just the Kubernetes/Docker Flow Runners at the moment
@Michael_Adkins: There have been a lot of difficulties integrating with AWS’s API in 1.0 so we’re intending to implement it very carefully this time 
@Julian_Mehnle: What would you recommend for a greenfield project that’ll run on AWS, with a strong hesitance to adopt Kubernetes at this time? Create your own fleet of EC2 instances that run Docker?
(I’m working with @Matt_Delacour on this.)
@Kevin_Kho: Unless Michael corrects me, I think that is right. Agent on EC2 with Flows that use DockerFlowRunner
@Anna_Geller: @Julian_Mehnle in Prefect 2.0, this is even easier than that! Given that the same agent process can simultaneously deploy flow runs as Docker containers, subprocesses, or Kubernetes jobs, depending on which flow_runner type you attach to your DeploymentSpec. I have a rough draft of a tutorial for AWS EC2 agent and was planning to polish it more before publishing, but given you need it now, I will share it already:
@Julian_Mehnle: Thanks, that’s useful.
When would you say that ECS (not EC2) support is likely to be available in 2.0? Some time in 2022?
@Anna_Geller: Would you be open to use EKS on Fargate instead of ECS Fargate?
@Julian_Mehnle: Open, but less likely.
K8s introduces a whole other virtualization layer on top of a virtual data center that we’re skeptical of dealing with for the foreseeable future.
FWIW, we can absolutely run a small EC2 instance fleet for now.
@Anna_Geller: From the perspective of cost and convenience benefits, Fargate is definitely attractive, but in general ECS has a really convoluted API and it’s hard to build an interface that will match user’s expectations since everyone seems to want to use ECS differently 
Some want to use it with EC2, some with ECS data plane, then there are different capacity providers, complex networking setup that many users would want Prefect to infer automatically from a default VPC, while more advanced users want to build it via a custom VPC deployed with Terraform - a really tough problem tbh. That’s why we need more time to think about the right way of exposing that API to users.
In contrast, if you would use EKS on Fargate, it’s “just” Kubernetes in the end which has a standardized API we can support out of the box with KubernetesFlowRunner. And for that, we already have this awesome tutorial from Laura:
You could create an EKS on Fargate cluster using eksctl in a single command (LMK if you need help with that) and then follow the tutorial from Laura to deploy the execution layer
@Julian_Mehnle: Understood. We will likely start with a small fleet of EC2 instances running the Prefect agent. We’ll cross the K8s bridge if and when that becomes necessary.
(And, yes, we’re using Terraform to manage the underlying infrastructure.)