Set the job_template in Kubernetes

Help
,
1

Hi,

I am trying to set a custom job template in kubernetes, as suggested here (Worker - prefect-kubernetes) but I don’t understand where to do it. I can successfully set the variables in the template in my prefect.yaml by doing something like:

definitions:
  tags: &common_tags
  - k8s
  work_pool: &common_work_pool
    name: kubernetes-pool
    job_variables:
      image: '{{ build-image.image }}'
      namespace: my-namespace-default

But where can I set a custom template? It seems like it would be a property of the work-pool, but I can’t seem to figure out how to make it work.

We setup the prefect-server and prefect-worker using the Helm charts.

Thanks,

Matt

2

Update: So I see that the documentation says:

For advanced use cases, users can create work pools with fully customizable job templates. This customization is available when creating or editing a work pool on the 'Advanced' tab within the UI.

Is that the only way? I am able to make the changes I want to add nodeSelector and tolerations to the job template via the UI, but I’d really like to do it programmatically when creating a new work pool. Anyone have a suggestion on how to do this?

I’ve looked through the code examples here, prefect-kubernetes, but it does not seem that that is exactly what I need. It looks like I need to maybe create a KubernetesWorkerJobConfiguration but then I am not sure how I would apply that to a work pool.

Maybe I need to switch back to agents? It seemed that the trend (recommendation) was to move away from agents, so I’d figured I would try that, but maybe for certain things agents are the way to go?