How to Send Prefect Logs to Azure Application Insights

Help
1

You can send Prefect log output to Azure Application Insights using Microsoft’s Azure Log Handler extension for the Python OpenCensus SDK.

As of January 2023, this is the only Microsoft-supported method for sending your logs. Note, however, that OpenCensus is now deprecated in favor of OpenTelemetry. Microsoft’s supported log handler will likely change as a result, but the OpenCensus SDK extension should continue working for the foreseeable future.

Read on to learn how to send Prefect logs to Azure Application Insights.

Custom Prefect logging configuration

A logging.yml configuration file is the easiest way to customize Prefect logging. Use Prefect’s default logging configuration as a starting point by copying its text into a new file. Note that this log configuration requires Prefect 2.6.9 or above. If you are using an older version of Prefect 2, use this configuration as your starting point instead.

The first step in adding Azure logging is creating a new logging handler. The handlers section of the default logging.yml shows the existing log handlers: the console handler, and the orion handler that sends logs to the API:

handlers:

    # The handlers we define here will output all logs they receive by default
    # but we include the `level` so it can be overridden by environment

    console:
        level: 0
        class: prefect.logging.handlers.PrefectConsoleHandler
        formatter: standard
        styles:
            # styles omitted for brevity

    orion:
        level: 0
        class: prefect.logging.handlers.OrionHandler

Adding a Handler for Azure

First, install the opencensus-ext-azure package anywhere you want to send your Prefect logs to Azure. This may include servers where you run Prefect Orion or a Prefect agent, or containers that run your Prefect flows.

To install the package, run:

pip install opencensus-ext-azure

Then, in the handlers section of your custom logging.yml, add an entry for the Azure log handler:

handlers:

    # default handlers appear here; leave them as-is.

    azure:
        level: 0
        class: opencensus.ext.azure.log_exporter.AzureLogHandler
        connection_string: "<your_connection_string"

Finding your Connection String

You can find your connection string in the overview section of your app’s Application Insights page or by running the following CLI command:

az monitor app-insights component show --app <your app name> --resource-group <your resource group name>

If you’d prefer not to save your connection string in YAML, you can leave it out and instead store it in the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable before you run Prefect.

However, Microsoft does not consider the connection string a secret and recommends using Azure AD authentication to limit write access to Application Insights.

Adding the Handler to Loggers

After adding the handler, you must tell Prefect where to use it.

For example, to send log entries from flow and task runs, update the prefect.flow_runs and prefect.task_runs loggers like so:

loggers:
    # other loggers; leave as-is

    prefect.flow_runs:
        level: NOTSET
        handlers: [orion, azure]

    prefect.task_runs:
        level: NOTSET
        handlers: [orion, azure]

    # remaining loggers; leave as-is

Running Prefect with custom logging configuration

The final step is getting Prefect to use your logging configuration.

The easiest way is to put your logging configuration in a file named logging.yml in Prefect’s home directory. The default Prefect home directory varies by operating system:

  • /home/<your username>/.prefect on Linux
  • /Users/<your username>/.prefect on MacOS
  • C:\Users\<your username>\.prefect on Windows

Alternatively, you can keep your logging configuration elsewhere and run the following to tell Prefect where to look for logging configuration:

prefect config set PREFECT_LOGGING_SETTINGS_PATH=/path/to/your/logging.yml

As with the pip package mentioned earlier, you’ll must include the custom logging configuration anywhere you want to send Prefect logs to Azure.

Viewing Prefect Logs in Azure Application Insights

You can view the logs Prefect sends to Application Insights in the Azure Portal or via the Azure CLI.

Azure Portal

After you saving your logging configuration and running some flows, you can find your logs by opening your Azure Application Insights app in the Azure Portal, then navigating to Logs in the menu:

Screenshot 2023-01-14 at 3.49.36 PM

Then, view your log entries by querying the traces table:

Screenshot 2023-01-14 at 4.02.50 PM
Screenshot 2023-01-14 at 4.02.50 PM1308×516 35.8 KB

Azure CLI

You can also retrieve log entries from Application Insights via the Azure CLI:

az monitor app-insights events show --app <your app name> --resource-group <your resource group> --type traces
2 Likes
2

Hi Ryan, first off, great post!

Do you have any advice on how we could send logs to Azure if we’re running Prefect Cloud with AKS? Would we still use the logging.yaml file? If so, where should it live and how do we configure Prefect to use it? Thanks in advance!

1 Like
3

Hi, Rajesh!

You would still use logging.yml, and the way to do this on AKS would be to put logging.yml in your container image to ensure it is available anywhere you run an agent or flow. If you put logging.yml in the home directory of the user account that runs Prefect inside the container, Prefect will use it automatically.

Or, you can put it in a different directory and set the PREFECT_LOGGING_SETTINGS_PATH environment variable to the full pathname (including the filename) of your custom logging configuration file.

You’ll also need to install the opencensus-ext-azure package in any container where you want to send logs to Application Insights.

1 Like