Prefect colored console logging using rich

To colorize console logs like below:

Step 1:
Copy prefect/logging.yml at orion · PrefectHQ/prefect · GitHub to ~/.prefect/logging.yml)

Step 2:
pip install rich

Step 3:
Replace the handlers section’s classes with RichHandler


    console:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard

    console_flow_runs:
        level: 0
        class: rich.logging.RichHandler
        formatter: flow_runs

    console_task_runs:
        level: 0
        class: rich.logging.RichHandler
        formatter: task_runs

To achieve:

To drop duplicate info, you can either:

  • Update logging.yml again with updated format
    format: "%(levelname)-7s | Task run %(task_run_name)r - %(message)s"

  • You may also create a custom handler and use that:

from rich.logging import RichHandler

class CustomRichHandler(RichHandler):
    def __init__(self, *args, **kwargs):
        kwargs["extras"] = "markup"
        kwargs["rich_tracebacks"] = True
        kwargs["show_time"] = False
        kwargs["show_path"] = False
        kwargs["show_level"] = False
        kwargs["tracebacks_word_wrap"] = False
        super().__init__(*args, **kwargs)

Valid kwargs:

Args:
    level (Union[int, str], optional): Log level. Defaults to logging.NOTSET.
    console (:class:`~rich.console.Console`, optional): Optional console instance to write logs.
        Default will use a global console instance writing to stdout.
    show_time (bool, optional): Show a column for the time. Defaults to True.
    omit_repeated_times (bool, optional): Omit repetition of the same time. Defaults to True.
    show_level (bool, optional): Show a column for the level. Defaults to True.
    show_path (bool, optional): Show the path to the original log call. Defaults to True.
    enable_link_path (bool, optional): Enable terminal link of path column to file. Defaults to True.
    highlighter (Highlighter, optional): Highlighter to style log messages, or None to use ReprHighlighter. Defaults to None.
    markup (bool, optional): Enable console markup in log messages. Defaults to False.
    rich_tracebacks (bool, optional): Enable rich tracebacks with syntax highlighting and formatting. Defaults to False.
    tracebacks_width (Optional[int], optional): Number of characters used to render tracebacks, or None for full width. Defaults to None.
    tracebacks_extra_lines (int, optional): Additional lines of code to render tracebacks, or None for full width. Defaults to None.
    tracebacks_theme (str, optional): Override pygments theme used in traceback.
    tracebacks_word_wrap (bool, optional): Enable word wrapping of long tracebacks lines. Defaults to True.
    tracebacks_show_locals (bool, optional): Enable display of locals in tracebacks. Defaults to False.
    locals_max_length (int, optional): Maximum length of containers before abbreviating, or None for no abbreviation.
        Defaults to 10.
    locals_max_string (int, optional): Maximum length of string before truncating, or None to disable. Defaults to 80.
    log_time_format (Union[str, TimeFormatterCallable], optional): If ``log_time`` is enabled, either string for strftime or callable that formats the time. Defaults to "[%x %X] ".
5 Likes

Instead of creating a separate class, you can override the kwargs directly in the yaml

    console:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard
        show_time: False
        show_path: False
        show_level: False

    console_flow_runs:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard
        show_time: False
        show_path: False
        show_level: False
        formatter: flow_runs

    console_task_runs:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard
        show_time: False
        show_path: False
        show_level: False
        formatter: task_runs

3 Likes

Cool tip. I think we can only specify the formatter to be either standard or flow_runs right?

    console:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard
        show_time: False
        show_path: False
        show_level: False

    console_flow_runs:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard
        show_time: False
        show_path: False
        show_level: False

    console_task_runs:
        level: 0
        class: rich.logging.RichHandler
        formatter: standard
        show_time: False
        show_path: False
        show_level: False
1 Like

Not sure!

Also, here’s a PR here that is more focused on highlighting important text and also allows customizability of colors. I need time to add tests for it though.

1 Like

This just needs a review and any suggestions before merging.

2 Likes