If you want to perform a clean up to free up space on your Postgres database instance, there are a couple of ways you can approach it:
- you can manually delete old records, especially logs from the flow run table using
DELETE FROMin SQL,
- you can do the same in an automated fashion, e.g. some users have an actual flow that runs on schedule and purges old data from the database,
- alternatively, you can use the open-source pg_cron job scheduler for Postgres to schedule such DB administration tasks,
for Prefect 1.0: you can also do the same using GraphQL: you would need to query for flow run IDs of “old” flow runs using the
flow_runquery, and then execute
delete_flow_runmutation - there is a separate Discourse topic that shows how you can do that: How can I delete flow runs older than 30 days using GraphQL API to clean up database space in Prefect Server?,
- for Prefect 2.0: you can do the same using the REST API.
To be more proactive, you can also reduce the number of logs you generate by:
- generally logging less (only logging what’s needed)
- setting the log level to a lower category, e.g. instead of using DEBUG logs on your agent, switching to INFO may significantly reduce the amount of space consumed by logs in the database.