How can I free up Postgres database space?

If you want to perform a clean up to free up space on your Postgres database instance, there are a couple of ways you can approach it:

  • you can manually delete old records, especially logs from the flow run table using DELETE FROM in SQL,
  • you can do the same in an automated fashion, e.g. some users have an actual flow that runs on schedule and purges old data from the database,
  • alternatively, you can use the open-source pg_cron job scheduler for Postgres to schedule such DB administration tasks,
  • for Prefect 1.0: you can also do the same using GraphQL: you would need to query for flow run IDs of “old” flow runs using the flow_run query, and then execute delete_flow_run mutation - there is a separate Discourse topic that shows how you can do that: How can I delete flow runs older than 30 days using GraphQL API to clean up database space in Prefect Server?,
  • for Prefect 2.0: you can do the same using the REST API.

To be more proactive, you can also reduce the number of logs you generate by:

  • generally logging less (only logging what’s needed)
  • setting the log level to a lower category, e.g. instead of using DEBUG logs on your agent, switching to INFO may significantly reduce the amount of space consumed by logs in the database.

Hi can you please share the flow for prefect 2 where we can purge the databse

1 Like