Missing schedule after re-deploying flow

I’m using Prefect Cloud and have noticed a problem with scheduled flow runs not being executed when I re-deploy an existing deployment (typically via a CI pipeline). What happens is the following:

  1. I create and apply a new deployment via the Python API using Deployment.build_from_flow(name="test-flow", work_queue_name="production", flow=my_flow, storage=my_storage);
  2. I attach a cron schedule to the deployment via the web interface, for example "0 11,12,13 * * *";
  3. Shortly after, the upcoming scheduled flow runs will appear in the web interface at 11:00, 12:00, 13:00;
  4. Next, I re-deploy by running the Python Deployment.build_from_flow again;
  5. The scheduled runs disappear from the web interface;
  6. At 11:00, I would expect the first scheduled run to occur. Instead, the scheduled runs re-appear in the web interface at this point, starting from 12:00;
  7. At 12:00 the job will run.

I have a CI pipeline that will redeploy all my jobs when code changes are made, and am noticing that this behaviour causes some flow runs to get skipped.

Have you tried deploying using CLI? this makes things much easier, especially in CI/CD

schedule should be unaffected by recreating a deployment (unless it changed)

btw you don’t need to recreate a deployment if e.g. only Docker image changed, you could only modify the image value on the block:

ecs = ECSTask.load("default")
ecs.image = "new_image:tag"
ecs.save("default", overwrite=True)

Thanks @anna_geller - this prompted me to do some more experimentation. I found:

  • The issue still happens when deploying with the CLI;
  • But it only happens if I use Azure for the storage block - I’ve made identical tests with AWS S3 and it works fine.

I’ve submitted a GitHub issue with steps to reproduce over at Missing schedule after redeploying flow using Azure storage · Issue #7657 · PrefectHQ/prefect (github.com).

1 Like

thanks so much for the issue, great writeup

1 Like