Launch Week Day 4 is not over yet! We've just released Customer Success Recipes and Prefect 2.0b2!

Announcements
, , , , , , , , , , , , , , , , , , , , , , , ,
1

Hi Prefectionists! :wave:

The launch week announcements for today are not over yet!

1. Customer Success Recipes :receipt:

Have you ever wondered how can you solve one of the following problems?

  • Deploy a Prefect agent on AWS EC2, EKS, and ECS using Terraform templates
  • Ensure repeatable deployments across various environments (dev, staging, production) using Infrastructure as Code
  • Build a CI/CD pipeline to automatically register Prefect flows upon closed pull requests using GitHub Action workflows
  • Integrate Prefect with Airbyte for EL and dbt for T (Yes, it stands for Extract - Load - Transform - a very common pattern in data engineering workflows)

All of the above points reflect extremely common use cases that nearly every new Prefect user needs to figure out on their own when getting started with Prefect. Since our mission is to eliminate negative engineering, we want to eliminate that burden and make it easy for you to implement best engineering practices when using Prefect. Therefore, we’re extremely excited to release something we’ve been working on for several months in the Customer Success Team - the Customer Success Recipes!

All of the recipes can be found in this public repository:

Shoutout to @George_Coyne, @jamiedick and @nate who put this together. This repository is a real treasure with many common flow deployment patterns and Prefect integrations. Don’t be fooled by the repository description:

Snippets and templates representing common Customer Success patterns

This extremely modest description :point_up: is a complete understatement with respect to how much value it provides! Make sure to check it out and star it! :star:

There are currently 14 recipes in there as of today, with plans to add many more over the coming weeks. Please feel free to request specific recipes via GitHub Issues. We also welcome contributions from your side if you are interested in collaborating on some of those efforts!

Here is a full list of available recipes and the roadmap ahead as of today:

1.1 AWS recipes

1.2 Prefect-oriented recipes

  • Airbyte Orchestration: kicking off Airbyte syncs with a Prefect flow
  • Using S3 Storage: using S3 to register flows as scripts
    • demo project: how to structure a repo where you need custom Python libraries
    • s3 to snowflake: basic ETL template using an S3 source and snowflake destination
  • Secret Import: Import your secrets from your local ~/.prefect/config.toml into your active Prefect Cloud tenant.

1.3 Roadmap of upcoming Customer Success Recipes

  • DBT orchestration
  • AWS Batch deployments
  • Orchestrating AWS batch with Prefect
  • Prefect agent deployments on GCP
  • CI registration to Prefect Cloud

See the open issues for a full list of proposed features (and known issues).

2. New Prefect 2.0b2 release

Thank you for jumping into Prefect 2.0b1 with such enthusiasm! :heart: Based on the feedback we’ve heard so far, the team has cut a release with some minor enhancements:

  • Docker flow runners can connect to local API applications on Linux without binding to 0.0.0.0
  • Adds with_options method to flows allowing override of settings e.g. the task runner.

…and a few fixes as well:

  • The CLI no longer displays tracebacks on a successful exit.
  • Returning pandas objects from tasks does not error.
  • Flows are now listed correctly in the UI dashboard.

You can learn more in the release notes.

Keep the feedback coming - it’s the fuel that drives Prefect 2.0’s development!

Missed previous LAUNCH WEEK announcements? Here’s a summary :point_down:

Happy engineering! :technologist: