This Discourse topic is meant to be used as a comprehensive list of resources pointing you to the right documentation or a tutorial. Depending on a different cloud provider or on-premise infrastructure requirements, you may need a different way of deploying Prefect 2.0.
Prefect 2.0 was designed to be a lightweight product that can simultaneously be used as an ephemeral orchestration API for local development (with embedded SQLite database), as well as a product that can scale to massive workloads when running it in a distributed fashion e.g. with Cloud 2.0.
https://discourse.prefect.io/t/running-prefect-2-with-docker-compose/1516
VIDEO
These instructions are accurate as of Prefect 2.0b5
Create a VM on Cloud Provider of Choice
Open port 4200 ingress on the VM from 0.0.0.0/0 (all traffic) as well as HTTP in.
pip install prefect 2.0
Set the UI_API_URL with :
prefect config set PREFECT_ORION_UI_API_URL="http://<external-ip>:4200/api"
Start Orion with:
prefect orion start --host 0.0.0.0
From local machine, configure to hit the API with:
prefect config set PREFECT_API_URL="http://<external-ip>:4200/api"
The remote UI will…
View in #show-us-what-you-got on Slack
[Paco_Ibañez] @Paco_Ibañez: Hello! I have being exploring Orion this week and I have put together a docker-compose to run Prefect 2.0 locally. I have tested it on mac and linux. The script runs prefect server, postgres, a docker agent, and minio to store flows. The start script configures a work queue and an example flow is included. Sharing in case someone finds it useful: https://github.com/fraibacas/prefect-orion
GitHub: GitHub - fraibacas/prefect-ori…
I’m trying to spin up a local prefect orion server for testing/development.
here’s the compose file I’ve come up with:
version: "3.7"
services:
postgres-orion:
image: "postgres:11"
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- postgres-orion-volume:/var/lib/postgresql/data
networks:
- prefect-orion
restart: "always"
command:
- "postgres"
- "-c"
…
Remember that there is an always-free tier - you can sign up and test out the platform. No trial accounts and no credit card are required.
Prefect 2.0 was designed to be a lightweight product that can simultaneously be used as an ephemeral orchestration API for local development (with embedded SQLite database), as well as a product that can scale to massive workloads when running it in a distributed fashion e.g. with Cloud 2.0.
https://discourse.prefect.io/t/should-i-start-with-prefect-2-0-orion-skipping-prefect-1-0/544
Welcome to Prefect! https://prefect.io/getting-started/overview/
quick start guide including instal…
2 Likes
thanks for the list of resources, it helped me get started on my experiment trying to self host Prefect on Google Compute Engine with terraform.
here’s a link to my repo if anyone’s interested: GitHub - VeraZab/elt-template: with terraform, GCP, Prefect and dbt