I want to use Kubernetes Secret as an environment variable when running a flow with Kubernetes job.
When deploying flow, I tried to override the default job template using job_variables
as follows, but it did not work.
Python source code(example_flow.py):
from prefect import flow
@flow(name="example-flow")
def example_flow():
#...something
if __name__ == "__main__":
example_flow.deploy(
name="example",
work_pool_name="kubernetes_work_pool", # Infrastructure Type is kubernetes
#...something
job_variables={
"env": {
"name": "DB_PASSWORD",
"valueFrom": {
"secretKeyRef": {
"name": "db",
"key": "password",
},
},
}
},
)
Output:
> python ./example_flow.py
...
prefect.deployments.runner.DeploymentApplyError: Error creating deployment: <ValidationError: "{'secretKeyRef': {'name': 'db', 'key': 'password'}} is not of type 'string'">
This seems to be due to the fact that in the base job template, the type of additionalProperties
for env
is a string.
"env": {
"type": "object",
"title": "Environment Variables",
"description": "Environment variables to set when starting a flow run.",
"additionalProperties": {
"type": "string"
}
Is there a way to get around these issues and use Kubernetes Secrets?
Are there any plans to release a feature for this?