Orchestrate a Data Science Project in Python With Prefect - by Khuyen Tran

A blog post showing how to get started with Prefect step-by-step.

Key points:

  • discussing the problem that Prefect solves,
  • creating tasks and flow,
  • visualizing and running the flow locally,
  • adding parameters,
  • using Prefect CLI to run parametrized flows,
  • registering the flow and a local agent to Prefect Cloud
  • showing the difference between default and custom parameters,
  • caching task inputs,
  • persisting task run results via checkpointing,
  • running dependent flows,
  • scheduling and logging.