Pass env vars from Kubernetes agent to job

Help
1

Hi,
is there a way to safely pass environment variables defined on the agent running in Kubernetes to
the Kubernetes Job Infrastructure?

I would love to define for instance DB_PASSWORD on the agent and pass that to each job. I know i can define env vars on the K8sJob Infrastructure level or via Secret, but that does not sound ideal for my use case.

Thanks for any help, cheers

2

Great question! The right way to do it is by using not environment variables, but Secret blocks

3

Can you please share an example for this pattern? Currently, I am trying to pass the env variables PREFECT_API_URL & PREFECT_API_KEY from the Agent (running in GCP Cloud Run) to the GCP Vertex job when agent launches the job. The agent already has those environment variables set.

To launch the Vertex job, I currently create a block like this but obviously this is not a good pattern since the API_KEY is visible.

from prefect_gcp.aiplatform import VertexAICustomTrainingJob

  block = VertexAICustomTrainingJob(
     image="us.gcr.io/..."
      # ...
      env={
          "PREFECT_API_URL": "SECRET_HERE",
          "PREFECT_API_KEY": "SECRET_HERE"
      }
  )
  
 block.save("vertex-job-test", overwrite=True)

If I wanted to use Secret blocks here, can I reuse the “VertexAICustomTrainingJob” infra block somehow? An example will be really helpful.