Prefect 1.0->2.0 local flow runner migration

Archive
, , , ,
1

Prefect 1.0 → 2.0 migration local flow runner

Summary

Prefect can run jobs locally. In Prefect 1.0, this could be achieved with either the UniversalRun config or with a LocalRun config. The local agent allows for greater granularity in flow configurations compared to the universal run. For example, environment variables and working directory path configs are exposed.

Video Link

https://share.descript.com/view/lmkq2F64yEX

What is Staying the Same

  • The main components of a Prefect flow will remain the same: you have tasks and flows. Tasks are essentially Python functions, while flows apply workflow logic to those tasks.
  • We still run the flow locally

What is Different

  • flow is imported with a lowercase ‘f’ rather than uppercase, and now has a decorator: @flow
  • there is no longer a concept of flow runners. Prefect 2.0 affords some of the customizability offered with a local flow runner, replacing labels with tags and using environment variables to override default configurations.

How to Convert from 1.0 to 2.0

  1. Install Prefect version 2.0+
  2. Replace from prefect import Flow with from prefect import flow
  3. Optionally, configure your flow with tags and environment variables
  4. You no longer call run on a Flow object, but call the flow as if its a function(see video)

Example 2.0 flow

from prefect import flow, task

@task
def do_something(x: int) -> int:
    print(x**2)
    return x**2

@flow
def my_flow(tags=["dev"]):
    do_something(5)

if __name__ == "__main__":
    my_flow()
1 Like