Filtering Logs from showing up in Prefect Cloud

Archive
1

Some users want to prevent certain logs from being logged by Prefect Cloud. For example, they may want to omit logs with certain stock tickers. This can be done by using a Python logger

import prefect
from prefect import task, Flow
import logging

# Creating a filter
class SecureFilter(logging.Filter):
    def filter(self, rec):
        if 'tes' in rec.msg:
            return 0
        return 1

@task
def abc():
    logger = prefect.context.get("logger")
    logger.addFilter(SecureFilter())
    logger.info("teting")        # gets shows
    logger.info("testing")       # gets hidden
    return 1

@task
def bcd():
    logger = prefect.context.get("logger")
    logger.addFilter(SecureFilter())
    logger.info("teting")        # gets shows
    logger.info("testing")       # gets hidden
    return 1

with Flow("test") as flow:
    abc()
    bcd()

flow.run()
1 Like
2

Note this filter happens at the logging level. In order to have the logs show up locally but not be sent to cloud, the filter can be attached at the handler level to the Cloud Handler.