The explanation below is a copy from this GitHub issue:
Many technologies can be used together, and in fact, it would be straightforward to run Prefect on top of a Celery backend (though we prefer Dask for the same purpose).
Celery is a system for executing work, usually in a distributed fashion. A “broker” submits work to a pool of workers, who run the task/job/function and indicate that they’ve finished. So Celery’s primary job is to take a function and run it. Dask isn’t exactly the same but similar enough for the purpose of this conversation.
Prefect, in contrast, is a workflow management system. Prefect’s primary job is to supply workflow semantics around your functions — things like what to do if they fail, how to handle logging, retries, conditional logic, responding to upstream results, complex dependencies, scheduling, etc. If all of that workflow logic “passes”, Prefect then runs your task on whatever backend you prefer, which is where Celery could come in to play.
Currently, Prefect ships with local, multiprocessing, threaded, and Dask distributed backends, and there’s an outstanding PR for a redis-queue backend. Celery would fit nicely as another execution backend.