Missing schedule after re-deploying flow

Archive
,
1

I’m using Prefect Cloud and have noticed a problem with scheduled flow runs not being executed when I re-deploy an existing deployment (typically via a CI pipeline). What happens is the following:

  1. I create and apply a new deployment via the Python API using Deployment.build_from_flow(name="test-flow", work_queue_name="production", flow=my_flow, storage=my_storage);
  2. I attach a cron schedule to the deployment via the web interface, for example "0 11,12,13 * * *";
  3. Shortly after, the upcoming scheduled flow runs will appear in the web interface at 11:00, 12:00, 13:00;
  4. Next, I re-deploy by running the Python Deployment.build_from_flow again;
  5. The scheduled runs disappear from the web interface;
  6. At 11:00, I would expect the first scheduled run to occur. Instead, the scheduled runs re-appear in the web interface at this point, starting from 12:00;
  7. At 12:00 the job will run.

I have a CI pipeline that will redeploy all my jobs when code changes are made, and am noticing that this behaviour causes some flow runs to get skipped.

2

Have you tried deploying using CLI? this makes things much easier, especially in CI/CD

schedule should be unaffected by recreating a deployment (unless it changed)

btw you don’t need to recreate a deployment if e.g. only Docker image changed, you could only modify the image value on the block:

ecs = ECSTask.load("default")
ecs.image = "new_image:tag"
ecs.save("default", overwrite=True)
3

Thanks @anna_geller - this prompted me to do some more experimentation. I found:

  • The issue still happens when deploying with the CLI;
  • But it only happens if I use Azure for the storage block - I’ve made identical tests with AWS S3 and it works fine.

I’ve submitted a GitHub issue with steps to reproduce over at Missing schedule after redeploying flow using Azure storage · Issue #7657 · PrefectHQ/prefect (github.com).

1 Like
4

thanks so much for the issue, great writeup

1 Like