How to use KubernetesAgent on Azure AKS with virtual nodes?

Archive
, , , , ,
1

Problem

To run prefect jobs in azure Kubernetes (AKS) virtual nodes, besides adding the recommended nodeSelector/tolerations:

"nodeSelector": {
    "kubernetes.io/role": "agent",
    "beta.kubernetes.io/os": "linux",
    "type": "virtual-kubelet",
},
"tolerations": [
    {"key": "virtual-kubelet.io/provider", "operator": "Exists"},
    {"key": "azure.com/aci", "effect": "NoSchedule"},
],

container creation fails with:

Warning ProviderCreateFailed   pod/prefect-job-XXXXX-XXXXX   
ACI does not support providing args without specifying the command. 
Please supply both command and args to the pod spec.

The different command options stated in this topic did not work well:
https://linen.prefect.io/t/48087/topic

Solution

command": ["tini", "-g", "--"],

solves the issue and allows prefect jobs to run in aks virtual nodes.

Here is an example of a working job_template:

{
    "apiVersion": "batch/v1",
    "kind": "Job",
    "spec": {
        "template": {
            "spec": {
                "containers": [
                    {
                        "name": "flow",
                        "command": ["tini", "-g", "--"],
                    }
                ],
                "nodeSelector": {
                    "kubernetes.io/role": "agent",
                    "beta.kubernetes.io/os": "linux",
                    "type": "virtual-kubelet",
                },
                "tolerations": [
                    {"key": "virtual-kubelet.io/provider", "operator": "Exists"},
                    {"key": "azure.com/aci", "effect": "NoSchedule"},
                ],
                "imagePullSecrets": [
                    {"name": "regcred"},
                ],
            },
        }
    },
}

Original post

Related topics