Log prefect logs to sql server table

Archive
1

I would like to add the ability to send prefect logs to a sql server table. Is there a way to accomplish this?

2

Hey, a couple of questions first:

  1. Are you on Prefect Cloud or self-hosted?
  2. If self-hosted, do you use SQLite or Postgres?
  3. What Prefect version do you use?
  4. Why would you need those logs to be in SQL Server?

Generally, there are a couple of ways to do that, depending on how fast you would need to get that data:

  1. Redirect logs
  2. Do database replication to get the entire logs table to SQL Server
  3. Use an EL tool such as Airbyte/Fivetran to run data replication
3

I’m assuming its self hosted. To my knowledge we are just using the prefect library. We are using prefect 2.2.0 and SQL Server. Ultimately we just want the logs stored so that we can go view them if we encounter an error. We will run our scripts on a server that has a monitoring service that will send emails to us if there is an error but we want the ability to store the logs if an error does occur so we can go back and see what the issue was.

4

You can do that from the Prefect UI directly - the UI is really excellent, I highly encourage you to give it a try and have a look, you’re gonna love that experience of inspecting the logs this way

Also here, Prefect already has a Notifications feature allowing you to get notified when any flow run fails - check out: