Hi everyone,
I am executing a flow on several computers using DaskTaskManager on prefect 2.3.1.
For the moment I test on two computer :
- one has the dask-scheduler and a dask-worker scheduled to the scheduler
- the other one only has a dask-worker scheduled to the scheduler too
Currently, I am only trying a simple code I found on this website :
from dask.distributed import Client
from prefect import flow, task
from prefect_dask import DaskTaskRunner
@task
def say_hello(name):
print(f"hello {name}")
@task
def say_goodbye(name):
print(f"goodbye {name}")
@flow(task_runner=DaskTaskRunner(address="tcp://X.X.X.X"))
def greetings(names):
for name in names:
say_hello.submit(name)
say_goodbye.submit(name)
However, when I run the code using one of the two computer, only the corresponding computer can do the tasks. The other one is returning this kind of error :
Exception: 'PrefectHTTPStatusError("Client error \'404 Not Found\' for url \'http://ephemeral-orion/api/task_runs/a158be60-8b91-49d8-ab7e-46a5969116d6/set_state\'\\nResponse: {\'exception_message\': \'Task run with id a158be60-8b91-49d8-ab7e-46a5969116d6 not found\'}\\nFor more information check: https://httpstatuses.com/404")'
and
distributed.protocol.pickle - INFO - Failed to deserialize
Thank you in advance !