Prefect Orion: Our Second-Generation Workflow Engine - by Jeremiah Lowin

The blog post announcing Orion: Prefect’s second-generation workflow engine.

Key points from the article

  • Orion is released as an open-source technical preview
  • Orion ships with a built-in dedicated orchestration API separating the orchestration engine from the code being orchestrated
  • Moving the engine behind an API allows Orion to become language-agnostic in the future
  • Instead of only providing workflows as code, Orion embraces “code as workflows” acknowledging that the user’s code is already the best representation of the workflow and significantly removing any unnecessary translation layer
  • Orion makes initial user onboarding a breeze, meets the users where they are and allows incremental adoption
  • Orion enables dynamic DAG-free workflows
  • Native code can be run side-by-side with fully orchestrated workflow steps allowing running any code within your flows, including Python-native if/else statements, for and while loops, and much more
  • Integration with Pydantic, FastAPI, asyncio, RRule, Sqlite and Dask
  • The rules engine: orchestration as a service - describes the role of an orchestrator and state transitions in Orion
  • Innovations in the UI - the new Orion Dashboard is fast, provides themes for colorblind users, and radial schematic to visualize workflows at any scale
  • Orion offers a new Data API for persisting arbitrary workflow results
  • Relationship to other Prefect products