What are the components of Prefect 2.0 architecture?

Prefect 2.0 was designed to be a lightweight product that can simultaneously be used as an ephemeral orchestration API for local development (with embedded SQLite database), as well as a product that can scale to massive workloads when running it in a distributed fashion e.g. with Cloud 2.0.

Docs

This documentation :point_down: should serve as your main source about the Prefect orchestration layer explaining all components and how they work together:

Consider everything else in this Discourse topic as supplementary material on top of that :point_up: documentation.

Main components of a single Orion instance

  1. Postgres or SQLite database used as a stateful backend system
  2. Orion API server
  3. REST API services used for orchestration
  4. UI

The API communicates requests between the server and UI, Python and CLI clients. The database is the only stateful component.

Managing multiple environments (dev, staging, production)

A single Prefect 2.0 installation can communicate with multiple Orion APIs, which can be helpful to switch between development, staging, and production environments.

To switch between those, you can use:

prefect profile use dev # or prod or any other profile

As long as each profile points to a different PREFECT_API_URL, you can easily switch between multiple instances.

Diagrams showing how various abstractions work together

Big picture

Detailed diagrams

image

image

Basic Deployment Structure

Creating Deployments

Multiple Deployments of the same flow

Triggering flow runs from deployments

Persisting flow run logs

Orion UI

Work queue creation

Tags & work queues

2 Likes

Related Slack Discussion about HA setup

View in #prefect-community on Slack

Tom_Manterfield @Tom_Manterfield: Hey everyone! Can anyone tell me if the Orion API and/or Scheduler and/or UI support running multiple replicas? Is anyone running them in an HA config already?

Anna_Geller @Anna_Geller: Great question! One engineer from our team thinks about this problem a lot and we may address this in the second half of the year, but due to the complexity of doing it well it’s not on an immediate roadmap

in general, the OSS Prefect 2.0 orchestration layer will be easier to scale than e.g. Prefect 1.0 due to simpler architecture - you should be able to deploy multiple API servers talking to the same Postgres backend and load balance the incoming requests

We definitely welcome your input if you are willing to contribute and share your solution with the community

Tom_Manterfield @Tom_Manterfield: > You should be able to deploy multiple API servers talking to the same Postgres backend and load balance the incoming requests
To clarify, is this saying you can do that right now and it’s just the scheduler that needs work, or are you describing the target state for later development?
because if the API is already good then it solves my need, the scheduler I feel like I can get by with vertical scaling for quite a while.

Anna_Geller @Anna_Geller: I don’t want to give you a wrong answer, but generally speaking, a scheduler service is part of the Orion API server and you could in theory (to achieve HA) have multiple uvicorn servers and load balance the requests

what I meant with complexity is e.g. ensuring how those multiple replicas work together e.g. I know that we use idempotency keys to avoid duplicate work (the same run being picked up twice) but I don’t know whether this would be enough to prevent duplicate work when running multiple Orion servers

This is definitely a hard problem (as anything distributed/HA) and my main point is that I don’t want to give you a sense that you can just deploy multiple replicas, add a load balancer on top and you’re done, there is a lurking complexity here to ensure it all works well together
> I feel like I can get by with vertical scaling for quite a while.
you’re right that vertical scaling seems to be the best solution right now - especially ensuring that your database scales and performs well because this is the only stateful component

Tom_Manterfield @Tom_Manterfield: Okay, from the sounds of things you’ve at least baked in a concept of idempotency. I do appreciate the caution, definitely better than overconfidence in this area. I might just give it a shot in dev for a bit and see what happens.

I was under the impression I could run the scheduler and the API separately though. I saw the --no-scheduler and --scheduler flags and assumed I could run Orion as the API only and then one with the scheduler only.

Anna_Geller @Anna_Geller: Indeed, it only shows that you already know more about it than I do :clap: it seems you can run a separate Orion server without UI, scheduler, mark late runs and analytics services
if you give it a try, could you share your work somehow e.g. via a public repo or even a public Gist with README? would be a great way to push the topic forward and continue iterating on that together, I’m sure others from the community would benefit from that too and could use it to contribute as well

Tom_Manterfield @Tom_Manterfield: Anything I make that’s shareable I definitely will. Once I’ve got my current setup working nicely and proven in a prod like environment then I’m planning to wrap most of it in a Kubernetes operator and open source it.

Don’t want to do that until I know it actually works though as I don’t want to push out rubbish and have some poor unsuspecting dev pick it up.

Michael_Adkins @Michael_Adkins: All of the services can be run separately from the API to scale them individually
I think your biggest issue in a setup with replicas will be Postgres database connection limits

Tom_Manterfield @Tom_Manterfield: Good info, thanks. I guess I can pool them with pgbouncer to ease that. I’m more comfortable having that problem than I am having something I can’t allow to scale.
@Michael_Adkins Is it possible to run the scheduler without the API? I can’t actually see how that one would be done. I can run the API without anything else, but I can’t see a way to do an equivalent of prefect orion start --scheduler-only
Seems like I’d have to run one API standalone and then one with the scheduler, and just scale the standalone one for API traffic and accept the scheduler always comes with an API attached. I can live with that if so, but thought I’d check if I missed something.

Michael_Adkins @Michael_Adkins: You can call python -m "prefect.orion.services.scheduler"

Tom_Manterfield @Tom_Manterfield: Oh cool! Thanks!

Michael_Adkins @Michael_Adkins: And similarly for any of the other services. I don’t think logging will be set up correctly yet though as we’ve not polished this.