How to set up prefect server and local agent to make flow of flows


I am running workflows on a self hosted infrastructure, with the default prefect server (the one launched running prefect server start) and a separate local agent in version 1.2.2.
It all works well, until I have to create a flow of flows, then each task creating a flow fails with the following error:

Task 'create_flow_run': Exception encountered during task execution!
Traceback (most recent call last):
  File "/home/debian/.local/lib/python3.9/site-packages/prefect/engine/", line 880, in get_task_run_state
    value = prefect.utilities.executors.run_task_with_timeout(
  File "/home/debian/.local/lib/python3.9/site-packages/prefect/utilities/", line 468, in run_task_with_timeout
    return*args, **kwargs)  # type: ignore
  File "/home/cloud/.local/lib/python3.9/site-packages/prefect/tasks/prefect/", line 167, in create_flow_run
  File "/home/debian/.local/lib/python3.9/site-packages/prefect/client/", line 1000, in get_cloud_url
    tenant_slug = self.get_default_tenant_slug()
  File "/home/debian/.local/lib/python3.9/site-packages/prefect/client/", line 1027, in get_default_tenant_slug
    if == self.tenant_id:
  File "/home/debian/.local/lib/python3.9/site-packages/prefect/client/", line 262, in tenant_id
    self._tenant_id = self._get_auth_tenant()
  File "/home/debian/.local/lib/python3.9/site-packages/prefect/client/", line 153, in _get_auth_tenant
    raise ValueError("You have not set an API key for authentication.")
ValueError: You have not set an API key for authentication.

The weird thing is that the children flows are created and ran without issues, even with that error.
But I would rather have the parent flow reporting a success.
What do I need to do to make it work?

Best regards,

The API key is requested by the client when your agent assumes you are running flows with the Prefect Cloud backend. To switch to Server, either run:

prefect backend server

before starting the agent, or set the env variable on your agent:

export PREFECT__BACKEND="server"

More on that here:

I did have this line in the backend.toml file of the machine hosting the prefect agent.
Do I need to put it on the machine hosting the prefect server, and on the ones hosting the dask clusters?
(I am using a remote dask cluster, so all the tasks including the creation of the the subflow are run on those dask workers)

This is definitely missing on the machine from which you are running your agent/execution layer

I confirm this was the issue. Adding the prefect config.toml and backend.toml files on the dask workers solved it.
Thanks a lot!

1 Like