Feedback on cleaner logs

Archive
, ,
1

Sample Code:

from prefect import flow, task

@task
def print_task(x):
    print(x)

@flow
def a_flow():
    for i in range(0, 3):
        print_task.with_options(name=f"print_task_{i}")(i)

a_flow()

Output:

17:23:37.359 | INFO    | prefect.engine - Created flow run 'cream-spoonbill' for flow 'a-flow'
17:23:37.359 | INFO    | Flow run 'cream-spoonbill' - Using task runner 'ConcurrentTaskRunner'
17:23:37.378 | INFO    | Flow run 'cream-spoonbill' - Created task run 'print_task_0-63def00f-0' for task 'print_task_0'
17:23:37.399 | INFO    | Flow run 'cream-spoonbill' - Created task run 'print_task_1-66cd300d-0' for task 'print_task_1'
0
17:23:37.431 | INFO    | Flow run 'cream-spoonbill' - Created task run 'print_task_2-4d6c89dd-0' for task 'print_task_2'
1
2
17:23:37.710 | INFO    | Task run 'print_task_1-66cd300d-0' - Finished in state Completed(None)
17:23:37.759 | INFO    | Task run 'print_task_2-4d6c89dd-0' - Finished in state Completed(None)
17:23:37.764 | INFO    | Task run 'print_task_0-63def00f-0' - Finished in state Completed(None)
17:23:38.397 | INFO    | Flow run 'cream-spoonbill' - Finished in state Completed('All states completed.')

I personally feel like there’s a lot of clutter. I was wondering if:

  1. Instead of the following:
    17:23:37.378 | INFO | Flow run 'cream-spoonbill' - Created task run 'print_task_0-63def00f-0' for task 'print_task_0'
    For succint-ness could it be like:
    17:23:37.378 | INFO | Created run 'cream-spoonbill.print_task_0-63def00f-0'
    Flow run I don’t think needs to be repeated; the flow name can be concatenated with the task run name, and because the task run contains the task name, the task reference can be dropped?

  2. For Completed(None), can it simply by Completed if None?
    17:23:37.710 | INFO | Task run 'print_task_1-66cd300d-0' - Finished in state Completed(None)

2

Optimizing for less content here is tricky. These messages are designed to be customizable via the logging template and to be clear when you have concurrent flow and task runs.

  1. While we could roll the flow run name as a prefix for the log there, we’re then being misleading about what the task run name is. We’re also not clarifying if it is a subflow run or a task run that is being created.

  2. I’m down to remove the None when there is no message.

1 Like
3

I see; maybe a way to keep it succinct and inform whether it’s a subflow or task run is through color coding.

Another way is to keep all this information in DEBUG, but have a separate shorter format for INFO?