Add separate logging FileHandlers within Python code, specific to a flow

Show and Tell
1

Contributed by Tuoyi Zhao:

import prefect
from prefect import flow, task
import logging
from datetime import datetime

@task
def string_prep(ct):
    return ct.strftime("%H:%M:%S")

@flow()
def dynamic_parameter(ct=datetime.now()):
    #Add File Handler to both flow and task logger
    flow_logger = logging.getLogger('prefect.flow_runs')
    task_logger = logging.getLogger('prefect.task_runs')

    handler = logging.FileHandler(filename=<replace with your own log file path>)
    handler.setFormatter(logging.Formatter("%(asctime)s.%(msecs)03d | %(levelname)-7s | %(name)s - %(message)s"))

    flow_logger.addHandler(handler)
    task_logger.addHandler(handler)

    #Actual Flow Part
    general_logger = prefect.logging.get_run_logger()
    strings = string_prep(ct)
    general_logger.info('see see ' + strings)
4 Likes
2

For global settings, it’s recommended that you use the logging.yml file as described here:

2 Likes
3

Will subflows or tasks inherit the filehandlers from the main flow?

4

if you follow this, it should: