View in #prefect-community on Slack
@Matt_Yamkowy: Hi. I’m just starting to look at Prefect 2.0. Is there a suggested way of handling a waiting period mid flow? For example if I start a flow, and after an initial task I want to wait for 5 minutes before continuing, do I just sleep for 5 minutes in a task? What about if I wanted to wait for days?
@Kevin_Kho: You could do that with a sleep yeah. What is the use case for waiting days? Because maybe at that point to can trigger a sub flow scheduled for a later time
@Matt_Yamkowy: In some cases we would detect an alarm, then start gathering data, days later we’d check if the gathered data met certain criteria before continuing with tasks. So are you saying I could dynamically schedule a follow up flow to check on my gathered data?
@Kevin_Kho: Ah I guess not yet. We do have Client.create_flow_run_from_deployment here but I guess it doesn’t take in a scheduled start time yet. The equivalent in Prefect 1 does
@Matt_Yamkowy: Hmm, can you suggest a way to do this? or will it be coming as 2.0 comes out of beta?
@Kevin_Kho: it will be coming, for now though you’d either need to poll or use Prefect 1.0. I did leave an internal message to confirm and i’ll get back to you
@Matt_Yamkowy: Thanks, I appreciate it!
@Andrew_Huang: Or maybe I was wondering if you could set up a scheduled deployment every hour to check if there’s new data; if not, stop the flow run prematurely and recheck in another hour? to prevent clobbering, could also set concurrency limit = 1 Schedules - Prefect Docs
@Kevin_Kho: I think at that point, you can just use the task level retry to achieve it can it will be simpler
@Matt_Yamkowy: It could be possible to setup a schedule to check if any jobs are complete, but we could potentially have a hundreds of different flows so we’d have to plan it out well. I was hoping prefect would have something built in to handle it.
I know temporal can do async sleep built in, I was hoping there was something similar in prefect.
@Kevin_Kho: Not exactly the same issue, but Add start and end dates to schedules · Issue #5620 · PrefectHQ/prefect · GitHub is heavily related. Definitely in the list of things to do for Prefect 2.0
@Matt_Yamkowy: This is great.