By default, Prefect outputs its logs to the console and sends them to the Prefect API (either Orion or Prefect Cloud).
However, many logging utilities and aggregators only work with logs written to files. Fortunately, adding filesystem logging to Prefect requires only a few steps.
Custom Prefect logging configuration
A logging.yml configuration file is the easiest way to customize Prefect logging. Use Prefect’s default logging configuration as a starting point by copying its text into a new file. Note that this log configuration requires Prefect 2.6.9 or above. If you are using an older version of Prefect 2, use this configuration as your starting point instead.
The first step in adding filesystem logging is creating a new logging handler. The handlers section of the default logging.yml shows the existing log handlers: the console handler, and the orion handler that sends logs to the API:
handlers:
# The handlers we define here will output all logs they receive by default
# but we include the `level` so it can be overridden by environment
console:
level: 0
class: prefect.logging.handlers.PrefectConsoleHandler
formatter: standard
styles:
# styles omitted for brevity
orion:
level: 0
class: prefect.logging.handlers.OrionHandler
A production-ready filesystem log handler should rotate logs to prevent filling the disk with log entries. Fortunately, Python’s logging.handlers module includes two log handlers that manage log file rotation:
-
RotatingFileHandler— rotates logs based on log file size. -
TimedRotatingFileHandler— rotates logs by time period.
You can add either of these handlers to the default logging.yml by adding one of the following entries to the handers section:
Size based log rotation
file:
level: 0
class: logging.handlers.RotatingFileHandler
# Update filename to the location you want to store Prefect's log files.
# This location must be writable by the user account that runs Prefect.
filename: /var/log/prefect.log
# maximum 10MB per log file
maxBytes: 10485760
# keep the last 5 log files in addition to the current log file
backupCount: 5
formatter: standard
This handler caps log files at a specific size. Depending on your OS and machine setup, you may need to change filename to a location Prefect can write to. Note that you must provide the file’s full path; using ‘~’ as a shortcut to the current user’s home directory won’t work.
By default, this handler lets log files grow to 10MB. Adjust this to suit your needs by changing the maxBytes property. The backupCount property controls how many old log files the log handler keeps in addition to the current log file.
See the Python docs on RotatingFileHandler for additional configuration options. Any argument you can pass to the class constructor can be set in logging.yml.
Time based log rotation
file:
level: 0
class: logging.handlers.TimedRotatingFileHandler
# Update filename to the location you want to store Prefect's log files.
# This location must be writable by the user account that runs Prefect.
filename: /var/log/prefect.log
when: 'D'
interval: 1
backupCount: 7
formatter: standard
This handler rotates log files by time interval. It’s set to use one log file per day, and keep logs for 7 prior days.
See the Python docs to learn how to configure TimedRotatingFileHandler. It offers many options for changing the logging interval and even the time of day when logs roll over to a new file.
JSON log formatting
The standard Prefect log formatter is good at generating human-readable logs. But if you’re using a log aggregator or APM look Datadog or Splunk, you will need machine-readable JSON logs.
Prefect comes with a JSON log formatter. To use it, change the formatter: standard line in your file log handler to formatter: json.
Prefect’s JSON formatter was updated in Prefect 2.6.9 to ensure its output is compatible with Datadog and other log ingestion tools. If possible, use Prefect >= 2.6.9 if you need JSON-formatted logs.
Using the file log handler
Now that you’ve added a file log handler, it’s time to use it. To add file logging to one of Prefect’s loggers, add file to the handlers section of the logger entry, or add a handlers section if it does not exist.
For example, here’s how the loggers section of logging.yml would look if you add the file handler to the prefect.flow_runs, prefect.task_runs, and prefect.infrastructure loggers:
loggers:
prefect:
level: "${PREFECT_LOGGING_LEVEL}"
prefect.extra:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [orion]
prefect.flow_runs:
level: NOTSET
handlers: [orion, file]
prefect.task_runs:
level: NOTSET
handlers: [orion, file]
prefect.orion:
level: "${PREFECT_LOGGING_SERVER_LEVEL}"
prefect.client:
level: "${PREFECT_LOGGING_LEVEL}"
prefect.infrastructure:
level: "${PREFECT_LOGGING_LEVEL}"
handlers: [file]
uvicorn:
level: "${PREFECT_LOGGING_SERVER_LEVEL}"
fastapi:
level: "${PREFECT_LOGGING_SERVER_LEVEL}"
Running Prefect with custom logging configuration
The final step is getting Prefect to use your logging configuration.
The easiest way is putting your logging configuration in a file named logging.yml in Prefect’s home directory. The default Prefect home directory varies by operating system:
-
/home/<your username>/.prefecton Linux -
/Users/<your username>/.prefecton MacOS -
C:\Users\<your username>\.prefecton Windows
Alternatively, you can keep your logging configuration elsewhere and run the following to tell Prefect where to look for logging configuration:
prefect config set PREFECT_LOGGING_SETTINGS_PATH=/path/to/your/logging.yml
Note that you’ll need to use your custom logging configuration anywhere you want Prefect to log to the filesystem. This may include servers where you run Prefect Orion or a Prefect agent, or containers
Testing file logging
With setup complete, all that’s left is running a flow to ensure file logging works.
Run a Prefect flow, and then check the file you set in your handler’s filename. The output should look similar to:
11:43:44.247 | INFO | Flow run 'orthodox-badger' - Created task run 'my_task-20c6ece6-0' for task 'my_task'
11:43:44.248 | INFO | Flow run 'orthodox-badger' - Submitted task run 'my_task-20c6ece6-0' for execution.
11:43:44.255 | INFO | Flow run 'orthodox-badger' - Created task run 'my_task-20c6ece6-4' for task 'my_task'
11:43:44.256 | INFO | Flow run 'orthodox-badger' - Submitted task run 'my_task-20c6ece6-4' for execution.
11:43:44.284 | INFO | Flow run 'orthodox-badger' - Created task run 'my_task-20c6ece6-3' for task 'my_task'
11:43:44.284 | INFO | Flow run 'orthodox-badger' - Submitted task run 'my_task-20c6ece6-3' for execution.
11:43:44.292 | INFO | Task run 'my_task-20c6ece6-0' - Finished in state Completed()
11:43:44.298 | INFO | Task run 'my_task-20c6ece6-4' - Finished in state Completed()
11:43:44.316 | INFO | Task run 'my_task-20c6ece6-3' - Finished in state Completed()
11:43:44.328 | INFO | Flow run 'orthodox-badger' - Created task run 'my_task-20c6ece6-1' for task 'my_task'
11:43:44.328 | INFO | Flow run 'orthodox-badger' - Submitted task run 'my_task-20c6ece6-1' for execution.
11:43:44.337 | INFO | Flow run 'orthodox-badger' - Created task run 'my_task-20c6ece6-2' for task 'my_task'
11:43:44.338 | INFO | Flow run 'orthodox-badger' - Submitted task run 'my_task-20c6ece6-2' for execution.
11:43:44.366 | INFO | Task run 'my_task-20c6ece6-1' - Finished in state Completed()
11:43:44.370 | INFO | Task run 'my_task-20c6ece6-2' - Finished in state Completed()
11:43:44.384 | INFO | Flow run 'orthodox-badger' - Finished in state Completed('All states completed.')