Py.warning logging raises MissingContextError

TomGoBravo · September 13, 2022, 8:34pm

For my first project with Prefect I’m moving a script that originally was run directly in a shell terminal (a flask and sqlalchemy cli) to be run within Prefect so other people can easily trigger it and see the results. The script calls the Python standard ‘warnings.warn’ to report some problems. When these didn’t appear in the Orion UI logs I changed the deployment to include "env.PREFECT_LOGGING_EXTRA_LOGGERS": "py.warnings" but now runs fail with the error:

Traceback (most recent call last):
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/logging/handlers.py", line 273, in emit
    self.get_worker(profile).enqueue(self.prepare(record, profile.settings))
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/logging/handlers.py", line 309, in prepare
    raise MissingContextError(
prefect.exceptions.MissingContextError: Logger 'py.warnings' attempted to send logs to Orion without a flow run id. The Orion log handler can only send logs within flow run contexts unless the flow run id is manually provided.

Logs, the Prefect Way - Prefect made me think I need only list extra loggers and they’d work, but that doesn’t seem to be happening for me. What am I missing?

Thank you for any ideas you can share!

anna_geller · September 14, 2022, 12:25am

Not sure what’s missing but the easiest way to set this up is via

prefect config set PREFECT_LOGGING_EXTRA_LOGGERS=‘logger1,logger2’

It must be a comma separated string if you have multiple loggers

TomGoBravo · September 14, 2022, 1:23am

I removed env.PREFECT_LOGGING_EXTRA_LOGGERS from the deployment infra_overrides and ran prefect config set PREFECT_LOGGING_EXTRA_LOGGERS='py.warnings' but it fails. Here’s more of the output:

18:11:30.536 | INFO    | prefect.agent - Completed submission of flow run '8e5a28ec-5acd-45d7-a635-14ac301bfbb5'
18:11:33.365 | ERROR   | Flow run 'gamma2-piscis-austrini-collapse' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/logging/handlers.py", line 273, in emit
    self.get_worker(profile).enqueue(self.prepare(record, profile.settings))
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/logging/handlers.py", line 309, in prepare
    raise MissingContextError(
prefect.exceptions.MissingContextError: Logger 'py.warnings' attempted to send logs to Orion without a flow run id. The Orion log handler can only send logs within flow run contexts unless the flow run id is manually provided.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/tmpws31g_3pprefect/tourist/scripts/dataflow.py", line 20, in <module>
    def run_fetch(source: scrape.Source):
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/flows.py", line 533, in flow
    Flow(
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/context.py", line 163, in __register_init__
    __init__(__self__, *args, **kwargs)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/flows.py", line 150, in __init__
    self.parameters = parameter_schema(self.fn)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/utilities/callables.py", line 159, in parameter_schema
    pydantic.create_model(
  File "pydantic/main.py", line 657, in pydantic.main.BaseModel.schema
  File "pydantic/schema.py", line 185, in pydantic.schema.model_schema
  File "pydantic/schema.py", line 590, in pydantic.schema.model_process_schema
  File "pydantic/schema.py", line 631, in pydantic.schema.model_type_schema
  File "pydantic/schema.py", line 258, in pydantic.schema.field_schema
  File "pydantic/schema.py", line 536, in pydantic.schema.field_type_schema
  File "pydantic/schema.py", line 935, in pydantic.schema.field_singleton_schema
  File "pydantic/schema.py", line 590, in pydantic.schema.model_process_schema
  File "pydantic/schema.py", line 640, in pydantic.schema.model_type_schema
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/warnings.py", line 109, in _showwarnmsg
    sw(msg.message, msg.category, msg.filename, msg.lineno,
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/logging/__init__.py", line 2245, in _showwarning
    logger.warning("%s", s)
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/logging/__init__.py", line 1489, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/logging/__init__.py", line 1624, in _log
    self.handle(record)
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/logging/__init__.py", line 1634, in handle
    self.callHandlers(record)
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/logging/__init__.py", line 1696, in callHandlers
    hdlr.handle(record)
  File "/home/thecap/.pyenv/versions/3.10.7/lib/python3.10/logging/__init__.py", line 968, in handle
    self.emit(record)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/logging/handlers.py", line 275, in emit
    self.handleError(record)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/logging/handlers.py", line 283, in handleError
    warnings.warn(exc, stacklevel=8)
TypeError: expected string or bytes-like object

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/engine.py", line 256, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/client.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/deployments.py", line 69, in load_flow_from_flow_run
    flow = await run_sync_in_worker_thread(import_object, str(import_path))
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 57, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(call, cancellable=True)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/utilities/importtools.py", line 193, in import_object
    module = load_script_as_module(script_path)
  File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/utilities/importtools.py", line 156, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'tourist/scripts/dataflow.py' encountered an exception
18:11:34.035 | INFO    | prefect.infrastructure.process - Process 'gamma2-piscis-austrini-collapse' exited cleanly.

If I set PREFECT_LOGGING_EXTRA_LOGGERS='' and look at the agent terminal (I’m running all this locally) I can see the logging statement that causes the MissingContextError:

18:19:32.602 | WARNING | py.warnings - /home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/utilities/callables.py:159: UserWarning: Callable extractor was excluded from schema since JSON schema has no equivalent type.
  pydantic.create_model(

anna_geller · September 14, 2022, 1:51am

Can you share your flow code that uses it? Redact for privacy

looks like you are running this outside of a Prefect flow and that’s why it’s failing to send logs to Prefect

TomGoBravo · September 14, 2022, 2:02am

Good point! Looking at the tracebacks I see def run_fetch(source: scrape.Source): which is my function decorated with @flow and then Flow( and File "/home/thecap/.pyenv/versions/tourist-3.10.7/lib/python3.10/site-packages/prefect/context.py", line 163, in __register_init__. Those lines are being run during flow initialization. pydantic/schema.py is adding a warning before the flow starts and .

Topic		Replies	Views
How do I get my logs into the Orion UI? Archive prefect-2-0 , logging	1	545	September 23, 2022
Profile and order of Python Imports not being recognized when running Deployments Help prefect-2-0 , local-development , deployment , troubleshooting	7	3032	January 23, 2023
Prefect throwing a traceback error Help	0	315	March 10, 2023
Prefect Agent -- 'C:\Program' is not recognized as an internal or external command on Windows Archive prefect-2-0 , agent	7	1013	October 31, 2023
Network issues when running Prefect Agent - httpx.ReadTimeout Archive prefect-2-0 , agent	9	1413	November 25, 2022

Py.warning logging raises MissingContextError

Related Topics