My flow won't stop running in the background

Archive
1

I am very new to prefect and I was testing a flow that creates rows in a postgres table when I accidentally started the flow before setting an end date in the schedule. Now the flow won’t stop running and is continually adding data to my table. I am using Vscode and ran the flow thru the interactive window which uses jupyter.

How do I stop my flow from running forever? I have tried overwriting the flow to include an end_date in the schedule, have also tried doing control + c in the terminal, killing my terminal, restarting my python in the interactive window, and restarting vscode. The flow must be running somewhere that I can’t find.

If anyone has any help I would greatly appreciate it. I can’t find any resources in here or the slack about this issue!

1 Like
2

If you delete the flow run from the UI, the process should be canceled as well.

3

Hi @anna_geller, I tried to delete a flow run from the UI, which I run through a Process deployment, but the process seems to keep going.
Also, does setting the flow run in the “Cancelled” state through the set_flow_run_state() API result in the process being stopped as well?
Thank you for your support!

4

gotcha, we are working on first-class flow run cancellation, no ETA but subscribe to the Topics tagged release-notes and this should be available pretty soon

1 Like
5

Great, thanks a lot! :slight_smile:

1 Like
6

@anna_geller I tested the 2.7.0 release and cancellation works as expected!

I only have an issue with task concurrency limits :frowning:
I have the following situation:

from prefect import task, flow

@task(tags=["TASK_TAG"])
def my_task():
    # long running task
    pass

@flow
def child_flow():
    my_task()

@flow
def parent_flow():
    # using .fn() should avoid any of the subflows deadlock issues described here: 
    # https://docs.prefect.io/concepts/tasks/#task-run-concurrency-limits
    child_flow.fn()

with TASK_TAG having a concurrency limit of 1.

When I cancel a run of the parent_flow while my_task is running, the task stays in the RUNNING state, while the flow goes in the CANCELLED state.

I believe this causes the TASK_TAG slot not being released, therefore re-running the flow results in the run being blocked while waiting for a TASK_TAG slot to be freed, which will never happen.
I noticed something similar while deleting a flow run as well.

Do you think what I’m describing makes sense? Thanks!

1 Like
7

this looks like a bug if this is what happens. Is this perhaps just UI issue? You could do a hard refresh to validate that.

Can you open a GitHub issue with a Minimal Reproducible Example if this persists?

8

Unfortunately, it is not a UI issue, I would create a dedicated GitHub issue, but I saw there is already #7732 that looks pretty related but pointing at sub-flows.
I left a comment there, but I can create a dedicated one if you find it appropriate.

EDIT: #7732 has been updated to address the issue on both sub-flows and tasks.

1 Like