Prefect Agent -- 'C:\Program' is not recognized as an internal or external command on Windows

John · December 29, 2022, 9:51pm

Using prefect tutorial, I set up my orion server, agent and create a python file as follow:

import sys
import prefect
from prefect import flow, task, get_run_logger
from prefect.orion.schemas.schedules import CronSchedule
from prefect.deployments import Deployment

@task()
def log_task(name):
    logger = get_run_logger()
    logger.info("Hello %s!", name)
    logger.info("Prefect Version = %s 🚀", prefect.__version__)

@flow()
def log_flow(name: str):
    log_task(name)

deployment = Deployment.build_from_flow(
    flow=log_flow,
    name="new deployment",
    description='this is a description',
    parameters={"name": "John"},
    schedule=(CronSchedule(cron="5 10,11 * * MON-FRI", timezone="America/Chicago")),
    work_queue_name="test",
)

When I just run a basic flow with task, the flow runs well and get completed

When I do deployment.apply() and wait for the scheduled flow to be executed it crashes and get the following issue:

Starting v2.7.4 agent with ephemeral API...

  ___ ___ ___ ___ ___ ___ _____     _   ___ ___ _  _ _____
 | _ \ _ \ __| __| __/ __|_   _|   /_\ / __| __| \| |_   _|
 |  _/   / _|| _|| _| (__  | |    / _ \ (_ | _|| .` | | |
 |_| |_|_\___|_| |___\___| |_|   /_/ \_\___|___|_|\_| |_|


Agent started! Looking for work from queue(s): test2...
15:03:55.391 | INFO    | prefect.agent - Submitting flow run '17c8ed90-3940-486e-95e5-f28fa4df60e5'
15:03:55.482 | INFO    | prefect.infrastructure.process - Opening process 'military-orca'...
15:03:55.510 | INFO    | prefect.agent - Completed submission of flow run '17c8ed90-3940-486e-95e5-f28fa4df60e5'
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
15:03:55.558 | ERROR   | prefect.infrastructure.process - Process 'military-orca' exited with status code: 1
15:03:55.584 | INFO    | prefect.agent - Reported flow run '17c8ed90-3940-486e-95e5-f28fa4df60e5' as crashed: Flow run infrastructure exited with non-zero status code 1.

I found two similar issues on (1) (2) but I’m still stuck.

Any help would be appreciated

anna_geller · December 30, 2022, 1:08am

can you try using CLI instead of doing it via python? prefect deployment build --help

John · December 30, 2022, 9:06pm

Using CLI gave me the same error. After trying on a different computer, I seems that prefect agent doesn’t recognize spaces in my python path. Creating a new venv partly help me solve the issue

Starting v2.7.5 agent with ephemeral API...

  ___ ___ ___ ___ ___ ___ _____     _   ___ ___ _  _ _____
 | _ \ _ \ __| __| __/ __|_   _|   /_\ / __| __| \| |_   _|
 |  _/   / _|| _|| _| (__  | |    / _ \ (_ | _|| .` | | |
 |_| |_|_\___|_| |___\___| |_|   /_/ \_\___|___|_|\_| |_|


Agent started! Looking for work from queue(s): test...
15:01:05.831 | INFO    | prefect.agent - Submitting flow run '37009a73-2d6e-4f39-8f21-fd9fd9ab1bef'
15:01:05.996 | INFO    | prefect.infrastructure.process - Opening process 'ginger-iguana'...
15:01:06.032 | INFO    | prefect.agent - Completed submission of flow run '37009a73-2d6e-4f39-8f21-fd9fd9ab1bef'
C:\Users\john\AppData\Local\Programs\Python\Python39\lib\runpy.py:127: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
15:01:09.241 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:///C:\Users\john\.prefect\orion.db
15:01:09.349 | DEBUG   | Flow run 'ginger-iguana' - Loading flow for deployment 'newDeployment'...
15:01:09.353 | ERROR   | Flow run 'ginger-iguana' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\engine.py", line 262, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\client\utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\deployments.py", line 181, in load_flow_from_flow_run
    flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\utilities\asyncutils.py", line 91, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\flows.py", line 772, in load_flow_from_entrypoint
    flow = import_object(entrypoint)
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\utilities\importtools.py", line 206, in import_object
    module = load_module(module_name)
  File "C:\Users\john\Documents\PythonVenv\Prefectv2\lib\site-packages\prefect\utilities\importtools.py", line 177, in load_module
    return importlib.import_module(module_name)
  File "C:\Users\john\AppData\Local\Programs\Python\Python39\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named '<input>'
15:01:09.402 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:///C:\Users\john\.prefect\orion.db
15:01:09.816 | INFO    | prefect.infrastructure.process - Process 'ginger-iguana' exited cleanly.

So now the flow start running but I don’t know how to solve this ModuleNotFoundError

anna_geller · December 31, 2022, 2:42pm

Search here on Discourse, we have a topic about it

John · January 4, 2023, 2:30pm

Thank you, I will check

Nowi5 · August 29, 2023, 4:44pm

were you able to find the solution? I have exactly the same issue.

prefect worker start --pool my-process-pool
Discovered worker type ‘process’ for work pool ‘my-process-pool’.
Worker ‘ProcessWorker ed68d26f-91ab-4943-8314-fcab150b0602’ started!
00:42:26.919 | INFO | prefect.flow_runs.worker - Worker ‘ProcessWorker ed68d26f-91ab-4943-8314-fcab150b0602’ submitting flow run ‘34d7141b-a888-402b-8b22-338c7dd00a76’
00:42:28.077 | INFO | prefect.flow_runs.worker - Opening process…
‘C:\Program’ is not recognized as an internal or external command,
operable program or batch file.
00:42:28.109 | ERROR | prefect.flow_runs.worker - Process 28172 exited with status code: 1
00:42:28.370 | INFO | prefect.flow_runs.worker - Completed submission of flow run ‘34d7141b-a888-402b-8b22-338c7dd00a76’
00:42:28.474 | INFO | prefect.flow_runs.worker - Reported flow run ‘34d7141b-a888-402b-8b22-338c7dd00a76’ as crashed: Flow run infrastructure exited with non-zero status code 1.

EmilRex · August 29, 2023, 4:54pm

@Nowi5 I believe this can happen when your path has a space in it. Does your flow code, virtualenv, or working directory contain spaces?

igomisp · October 31, 2023, 1:06pm

@EmilRex @Nowi5
I do have spaces in my paths and I have the same error considering the first of the path before the space as a command, raising an ERROR:

13:46:46.105 | INFO    | prefect.flow_runs.runner - Runner 'my-first-deployment' submitting flow run 'd24cf441-d012-45ae-a456-ac263a89b0d6'
13:46:46.258 | INFO    | prefect.flow_runs.runner - Opening process...
13:46:46.267 | INFO    | prefect.flow_runs.runner - Completed submission of flow run 'd24cf441-d012-45ae-a456-ac263a89b0d6'
"C:\Users\Nacho\OneDrive" no se reconoce como un comando interno o externo,
programa o archivo por lotes ejecutable.
13:46:46.277 | ERROR   | prefect.flow_runs.runner - Process for flow run 'astute-koala' exited with status code: 1
13:46:46.371 | INFO    | prefect.flow_runs.runner - Reported flow run 'd24cf441-d012-45ae-a456-ac263a89b0d6' as crashed: Flow run process exited with non-zero status code 1.
13:46:46.414 | INFO    | Flow run 'astute-koala' - Downloading flow code from storage at '.'

Is there a solution to override this issue? Or should I locate my file with a path with no spaces in it?