How do I suppress "Created task run" logs?

In my Jupyter notebook, I have to scroll very far down due to all the “Created task run” logs

I just want to see my own defined logging.info().

import xarray as xr
import matplotlib.pyplot as plt
from prefect import flow, task
from prefect.task_runners import DaskTaskRunner

@task
def plot(ds, time):
    logger = get_run_logger()
    logger.info(time)
    plt.figure()
    ds['air'].plot()
    plt.savefig(str(time)[:16])
    plt.close()


@flow(task_runner=DaskTaskRunner(cluster_kwargs={"n_workers": 4}))
def prefect_process():
    ds = xr.tutorial.open_dataset('air_temperature').isel(
        time=slice(0, 250))
    [plot(ds.sel(time=time), time) for time in ds['time'].values]

prefect_process()
1 Like

I remember there was a debate about this, and Michael made a very good point that this log is incredibly helpful in debugging issues, especially when using Dask and Ray task runners.

I was digging deeper into various log settings and thought that setting this may help:

prefect config set PREFECT_LOGGING_SERVER_LEVEL="CRITICAL"

But the task run log messages were still there.

Workaround

You may set all your log messages within your flow to a higher level than INFO e.g. WARNING - this way you can hide those task run logs and still show your important log messages.

Right, maybe the task_run / flow_run logs should be on their own level, e.g. logger.run_info(), a level in between logging.DEBUG and logging.INFO and Prefect would default to that, but the users can still use logger.info() normally and easily suppress the level logging.RUN_INFO if desired.

Here’s how to semi-suppress the flow run info

# jupyter syntax; use export if terminal
%set_env PREFECT_LOGGING_LOGGERS_PREFECT_FLOW_RUNS_LEVEL=DEBUG



import logging

import xarray as xr
import matplotlib.pyplot as plt
from prefect import flow, task, get_run_logger
from prefect.task_runners import DaskTaskRunner

@task
def plot(ds, time):
    logger = get_run_logger()
    logger.info(time)
    plt.figure()
    ds['air'].plot()
    plt.savefig(str(time)[:16])
    plt.close()


@flow(task_runner=DaskTaskRunner(cluster_kwargs={"n_workers": 4}))
def prefect_process():
    logger = get_run_logger()
    logger.setLevel(logging.WARNING)

    ds = xr.tutorial.open_dataset('air_temperature').isel(
        time=slice(0, 3))
    [plot(ds.sel(time=time), time) for time in ds['time'].values]

prefect_process()
1 Like

That’s an interesting idea! Would you want to bring this up tomorrow in product office hours? I think this would be the right place to discuss that.

I generally prefer having too many logs rather than not-enough logs so I really don’t mind the current behavior at all, but I can understand your frustration with the log overload here and there are certainly some ways to make it all more configurable.

Discussing with Michael here;

1 Like

The solution from Michael!

from prefect import flow, task
from logging import getLogger

logger = getLogger("my-logger")
logger.setLevel("INFO")


@task
def my_task():
    logger.info("Hello from the task")


@flow
def my_flow():
    logger.info("Hello from the flow")
    my_task()


my_flow()

Then run the flow with these environment variables:

PREFECT_LOGGING_LEVEL=WARNING PREFECT_LOGGING_EXTRA_LOGGERS=my-logger python flow.py

Results in only the custom loggers

12:40:19.285 | INFO    | my-logger - Hello from the flow
12:40:19.315 | INFO    | my-logger - Hello from the task
1 Like

That’s clean! Thanks for sharing here! :pray:

Here’s a better way, taken from Feature Request: Suppression of "task created" logs and UI log limitations · Issue #5952 · PrefectHQ/prefect · GitHub

After digging into logging more, I discovered the logging.yml file is pretty powerful, and I was able to remove “Created task run”, without dropping other logs!

Here’s my logging.yml file that I placed in ~/.prefect/logging.yml

# Prefect logging config file.
#
# Any item in this file can be overridden with an environment variable:
#    `PREFECT_LOGGING_[PATH]_[TO]_[KEY]=VALUE`
#
# Templated values can be used to insert values from the Prefect settings at runtime.

version: 1
disable_existing_loggers: False

formatters:
    simple:
        format: "%(asctime)s.%(msecs)03d | %(message)s"
        datefmt: "%H:%M:%S"

    standard:
        format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | %(name)s - %(message)s"
        datefmt: "%H:%M:%S"

    flow_runs:
        format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Flow run %(flow_run_name)r - %(message)s"
        datefmt: "%H:%M:%S"

    task_runs:
        format: "%(asctime)s.%(msecs)03d | %(levelname)-7s | Task run %(task_run_name)r - %(message)s"
        datefmt: "%H:%M:%S"

    json:
        class: prefect.logging.formatters.JsonFormatter
        format: "default"

filters:
    remove_created_task_run:
        (): __main__.RemoveCreatedTaskRun

handlers:

    # The handlers we define here will output all logs they receieve by default
    # but we include the `level` so it can be overridden by environment

    console:
        level: 0
        class: logging.StreamHandler
        formatter: standard
        filters: [remove_created_task_run]

    console_flow_runs:
        level: 0
        class: logging.StreamHandler
        formatter: flow_runs
        filters: [remove_created_task_run]

    console_task_runs:
        level: 0
        class: logging.StreamHandler
        formatter: task_runs
        filters: [remove_created_task_run]

    orion:
        level: 0
        class: prefect.logging.handlers.OrionHandler

loggers:
    prefect:
        level: "${PREFECT_LOGGING_LEVEL}"
        handlers: [console]
        propagate: no

    prefect.extra:
        level: "${PREFECT_LOGGING_LEVEL}"
        handlers: [orion, console]
        propagate: no

    prefect.flow_runs:
        level: NOTSET
        handlers: [orion, console_flow_runs]
        propagate: no

    prefect.task_runs:
        level: NOTSET
        handlers: [orion, console_task_runs]
        propagate: no

    prefect.orion:
        level: "${PREFECT_LOGGING_SERVER_LEVEL}"

    uvicorn:
        level: "${PREFECT_LOGGING_SERVER_LEVEL}"
        handlers: [console]
        propagate: no

    fastapi:
        level: "${PREFECT_LOGGING_SERVER_LEVEL}"
        handlers: [console]
        propagate: no

# The root logger: any logger without propagation disabled sends to here as well
root:
    # By default, we display warning level logs from any library in the console
    # to match Python's default behavior while formatting logs nicely
    level: WARNING
    handlers: [console]

Here’s the Python code:

import logging
from prefect import flow, task, get_run_logger


class RemoveCreatedTaskRun(logging.Filter):
    def filter(self, record):
        return 'Created task run' not in record.msg


@flow
def test_flow():
    test_task()


@task
def test_task():
    logger = get_run_logger()
    logger.info("abc")


test_flow()

Here’s the output!

13:00:39.210 | INFO    | prefect.engine - Created flow run 'turquoise-leech' for flow 'test-flow'
13:00:39.211 | INFO    | Flow run 'turquoise-leech' - Using task runner 'ConcurrentTaskRunner'
13:00:39.800 | INFO    | Task run 'test_task-c60a5fe2-0' - abc
13:00:39.931 | INFO    | Task run 'test_task-c60a5fe2-0' - Finished in state Completed()
13:00:40.056 | INFO    | Flow run 'turquoise-leech' - Finished in state Completed('All states completed.')
Completed(message='All states completed.', type=COMPLETED, result=[Completed(message=None, type=COMPLETED, result=None, task_run_id=f448446f-ffd9-48a4-b2bf-b1ab4533af54)], flow_run_id=c189df09-d06d-42d4-90e3-cbe357a7f63d)

The possibilities are endless!

class RemoveFinishedInState(logging.Filter):
    def filter(self, record):
        return 'Finished in state' not in record.msg
    remove_finished_in_state:
        (): __main__.RemoveFinishedInState
     ...
     filters: [remove_created_task_run, remove_finished_in_state]

References:
Prefect Logging: Logging - Prefect 2.0
Logging YAML: logging.config — Logging configuration — Python 3.10.5 documentation
Filters: Logging in python with YAML and filter - Stack Overflow

Hope this helps!

1 Like

Wow, this is really nice, thanks for sharing this solution! I love that you don’t have to reconfigure this every single time you add logger, but rather configure this only once - super clean

1 Like