This list of resources will point you to any topic of your interest regarding how to build a reliable flow deployment process suitable to your use cases, your data stack, and your team.
Recipes and documentation to reliably manage dataflow operations:
setting up your execution layer (your agents ) with Infrastructure as Code
creating remote storage and infrastructure blocks
building reliable deployments
testing your flows
setting up a CI/CD pipelines
best practices for managing dataflow operations
Here are some repository templates and resources to help you build a CI/CD pipeline for your flow deployments.
A single Prefect 2.0 installation can communicate with multiple Orion APIs, which can help you switch between development, staging, and production environments.
To switch between those, you can use:
prefect profile use dev # or prod or any other profile
As long as each profile points to a different PREFECT_API_URL, you can easily switch between multiple instances.
Each profile can point to:
a different Prefect Cloud workspace
a different Prefect Orion API endpoint
More information about…
[image]
Observable and robust real-time dataflows running on Serverless containers on AWS with a fully-automated deployment pipeline - sounds difficult? Prefect 2.0 makes it easy. Check our latest article.
if you want to jump straight into the code:
This post will describe how to run a Prefect flow in an Azure Function in response to new or updated files in Azure Blob Storage.
installed for your operating system installed and signed in .
The latest version of the Azure Functions Core Tools .
A Prefect Cloud acco…
This Discourse topic is meant to be used as a comprehensive list of resources pointing you to the right documentation or a tutorial. Depending on a different cloud provider or on-premise infrastructure requirements, you may need a different way of deploying Prefect 2.0.
The following diagram helps illustrate the relationships between different objects, including the underlying schema (at the time of writing).
[q]
This Discourse topic collects best practices and guidance around testing your dataflow.
4 Likes