How can I free up Postgres database space?

Archive
, , , , , ,
1

If you want to perform a clean up to free up space on your Postgres database instance, there are a couple of ways you can approach it:

  • you can manually delete old records, especially logs from the flow run table using DELETE FROM in SQL,
  • you can do the same in an automated fashion, e.g. some users have an actual flow that runs on schedule and purges old data from the database,
  • alternatively, you can use the open-source pg_cron job scheduler for Postgres to schedule such DB administration tasks,
  • for Prefect 1.0: you can also do the same using GraphQL: you would need to query for flow run IDs of “old” flow runs using the flow_run query, and then execute delete_flow_run mutation - there is a separate Discourse topic that shows how you can do that: How can I delete flow runs older than 30 days using GraphQL API to clean up database space in Prefect Server?,
  • for Prefect 2.0: you can do the same using the REST API.

To be more proactive, you can also reduce the number of logs you generate by:

  • generally logging less (only logging what’s needed)
  • setting the log level to a lower category, e.g. instead of using DEBUG logs on your agent, switching to INFO may significantly reduce the amount of space consumed by logs in the database.
2

Hi can you please share the flow for prefect 2 where we can purge the databse

1 Like