New release - Prefect 2.0.4 has just arrived! 🛬

Announcements
, , , ,
1

Deployment improvements

Based on your feedback, we added significant improvements to deployments:

  1. Prefect no longer generates manifest.json – many users gave us feedback that having two manifests (YAML for infrastructure and JSON for flow path configuration) was confusing. We simplified that. Now, there is a single YAML manifest that gets generated as a build artifact anytime you run prefect deployment build. You may version control that file or simply use it as a build artifact in your CI/CD (recipes for that are coming!).
  2. The local file system block is handled behind the scenes to streamline the local development experience.
  3. The infrastructure blocks have been optimized for modularity and composability. Instead of boilerplate code, you can now have a generic Process, DockerContainer or KubernetesJob block, and add custom overrides in your build CLI only for changes (overrides) you need to apply on a per-deployment level. For example, if you want to change the default log level for a given deployment to DEBUG (or change any other environment variable) or swap the Docker image for a different one, you can do all that using the new --override flag:
prefect deployment build flows/your_flow.py:flow_function_name \
-n deployment_name \ # name for this specific deployment
-t your_workqueue_name \ # tag corresponding to your work queue agent
-o deployments/deployment_manifest1.yaml \ # output: where to store your manifest
-ib process/dev \  # your generic parent Infrastructure Block to avoid boilerplate
--override env.PREFECT_LOGGING_LEVEL=DEBUG \ 
--override image=prefecthq/prefect:2-python3.9 \
-v your_github_sha  # version: for robust deployment versioning from CI/CD

Start custom flow runs from the UI

One of the most commonly requested features was the ability to start parametrized flow runs directly from the UI. You can now do that!From the deployment page, in addition to triggering an immediate flow run with default parameter arguments, you can also create a custom run allowing you to configure:

  • the run’s parameter values,
  • start time,
  • custom run name,
  • and more,

All that is possible while using otherwise the same deployment configuration (infrastructure, storage, etc.). The deployment itself will be unchanged and continue to generate runs on its regular schedule. See the attached screenshot to see how this works.

image
image1065×274 28.9 KB

image
image1386×836 48 KB

Further improvements and bug fixes

  • Timeout errors messages on state changes are more intuitive
  • Added debug level logs for task run rehydration
  • Added basic CLI functionality to inspect Blocks; more to come. Here is how it looks like:
prefect block ls # list blocks in the workspace 
prefect block type ls # list registered block types 
prefect block register --help # shows how to register new block types
  • Added support for filtering on state name to prefect flow-run ls
  • Refined autogenerated database migration output

Next steps

As always, to upgrade to the latest version:

pip install prefect -U

Cloud 2.0 is already up to date! :cloud:

Release notes on GitHub

Happy engineering!

2 Likes