RayTaskRunner freezes when writing data to a local directory

simon_mackenzie · April 6, 2022, 1:24pm

Thanks. Tried running your examples and they worked! However adapted them and it seems to be that it freezes when there are a lot of tasks with dependencies.

github.com/PrefectHQ/prefect

raytaskrunner freezes in some circumstances

opened 01:22PM - 06 Apr 22 UTC

simonm3

v2

Example code below. All tasks are submitted but only 5 get executed then it free…zes. Works for smaller number of tasks or if the tasks don't have the sleep. ``` from prefect import flow, task from prefect.task_runners import RayTaskRunner import shutil import os from time import sleep @task def task1(i): sleep(2) return i @task def task2(i): sleep(2) with open(f"temp/i{i}", "w") as f: f.write("xxx") return i @flow(task_runner=RayTaskRunner()) def ray_flow(): for i in range(25): x = task1(i) y = task2(x) if __name__ == "__main__": os.chdir("/v1") os.makedirs("temp", exist_ok=True) os.system("rm temp/*") ray_flow() ```

Topic		Replies	Views
How can I run tasks in parallel? Archive migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , infrastructure , multithreading , multiprocessing , basics	0	3921	January 31, 2022
Why is my function not running in parallel even with DaskTaskRunner / RayTaskRunner? Archive prefect-2-0 , dask , task-runner , parallel-processing , dask-task-runner , dask-annotations	14	1500	July 5, 2022
Edgecase Error: "Prefect Task run TASK_RUN_ID already finished" Archive prefect-2-0 , parallel-processing , ray , ray-task-runner	10	1246	December 1, 2022
After prefect>=2.0b9 to run in parallel, calling the submit method is required! Archive prefect-2-0 , dask , dask-task-runner , ray , ray-task-runner	0	492	July 20, 2022
Prefect 2.0 UI doesn't seem to show all task runs running on Dask in real-time Archive prefect-2-0 , dask , parallel-processing , dask-task-runner	20	1014	July 5, 2022

RayTaskRunner freezes when writing data to a local directory

Related Topics