Using a ShellOperation to execute a scrypt that utilizes multiprocessing

Help
, ,
1

Hi,

I have a task that uses ShellOperation to execute a script that is utilizing multiprocessing.
Here is an example.

test.py

from multiprocessing import Pool


def fx(x):
    y = x**2


def test():
    items = range(6)

    with Pool() as pool:
        pool.map(fx, items)


if __name__ == '__main__':
    test()

test_flow.py

from prefect import (
    flow, 
    task
)
from prefect_shell import ShellOperation


@task
def shell_task():
    with ShellOperation(
        commands=[
            'python test.py'
        ],
    ) as multiprocessing_operation:
        process = multiprocessing_operation.trigger()
        process.wait_for_completion()


@flow
def my_flow():
    shell_task()


if __name__ == '__main__':
    my_flow()

The script I’m trying to execute is ```rclone``

Thanks