AKS flow run example

I tried to follow this example and test a flow run.
How to deploy a flow to Kubernetes in Azure AKS with Prefect - Deployments & DevOps - Prefect Community

In this example, I used my local PC where the source of flow definition resides. So does the AKS agent need to be able to pull the flow from my local PC? ( My agent label is same as Flow label)
I am asking this question as I see an error when trying this example out.

Looking up flow metadata… Done
Creating run for flow ‘aks-flowrun’… Done
└── Name: ultramarine-camel
└── UUID: 4cb4d6d0-5ca2-4580-94a1-e11f23c8ba5b
└── Labels: [‘XYZ’]
└── Parameters: {}
└── Context: {}
└── URL: https://cloud.prefect.io/xxx/flow-run/xxx
Watching flow run execution…
└── 07:38:11 | INFO | Entered state : Flow run scheduled.
└── 07:38:20 | INFO | Entered state : Submitted for execution
└── 07:38:21 | INFO | Submitted for execution: Job prefect-job-xxxx
└── 07:38:22 | INFO | Entered state : Failed to load and execute flow run: 
ModuleNotFoundError(“No module named ‘C’”)

This won’t work with a remote agent because the Azure VM has no access to files on your local machine. You would need to use remote storage such as Azure blog storage.

And regarding your error message, check this page that provides a more detailed explanation of the issue:

Great information. It makes sense now. Thank you!.
I used Azure storage now in my Flow.

STORAGE = Azure(
    container="flows",
    connection_string_secret="AZURE_STORAGE_CONNECTION_STRING",   
    stored_as_script=True,
    blob_name=f"flows/{FLOW_NAME}.py",
)

I am able to register the flow and can see the flow in my storage account(encoded).

In Prefect Cloud UI, I can see my flow got registered. However my agent running on AKS cluster is not picking up and running the flow. The labels are same . Am I missing anything?

you are probably missing the run configuration. Can you share your full storage and run config and how you started your agent?

This is my flow:

import prefect
from prefect import task, Flow
from prefect.storage import Azure
from prefect.run_configs import KubernetesRun

STORAGE = Azure(
    container="flows",
    connection_string_secret="AZURE_STORAGE_CONNECTION_STRING",
    # storing as script with Azure would require uploading the file beforehand e.g. as part of CI/CD
    stored_as_script=True,
    # blob_name=f"flows/{FLOW_NAME}.py",
)
@task
def hello_task():
    logger = prefect.context.get("logger")
    logger.info("Hello world!")


with Flow("aks-flowrun", run_config=KubernetesRun(labels=["HUMPOC"])) as flow:
    hello_task()

if __name__ == "__main__":
    flow.register("SriniJ Prefect POC")

I can also see my Kubernetes agent in Prefect Cloud as healthy(checking for flows).

Does it mean your flow is stuck in a Scheduled state? If so, check this Discourse topic:

This is usually a label mismatch between the run config and your agent.

If this doesn’t help, can you share your Kubernetes agent deployment file?

I am running into an issue while sending the agent yaml file in my repsonse. I am getting a message that new users are allowed only 2 links per post.
The agent yaml I deployed to AKS cluster is the ouput i got from this:
prefect agent kubernetes install --rbac --key YOUR_API_KEY --label YOUR_LABEL > agent.yaml

I am expecting the Agent to run the flow and fail. This is what happens when I run the flow directly in Prefect Cloud. I expect this failure because AKS cluster/agent does not have network level access to the storage account.
However I expected the Agent to try running the flow automatically and showing me this error but apparently it never attempts to run my flow.

New update:
I tried with github as storage in my flow:
STORAGE = GitHub(
repo=“PrefectPOC”,
path=f"flows/{FLOW_NAME}.py",
access_token_secret=“GITHUB_ACCESS_TOKEN”,
)
but when i run it from Cloud UI, I get a 404
Failed to load and execute flow run: UnknownObjectException(404, {‘message’: ‘Not Found’, ‘documentation_url’: ‘GitHub REST API - GitHub Docs’}, {‘server’: ‘GitHub.com’, …

I created the secret in Prefect Cloud as well.

Did you install GitHub extras? The issue may be due to the missing GitHub package.

Prefect-base image has this package already - do you use PRefect base image? Can you share your Dockerfile and run configuration?

My agent is using prefecthq/prefect:1.2.0-python3.7 image.
My flow is using github storage for storage and kubernetes run for flow run.
with Flow(
FLOW_NAME,
storage=STORAGE,
run_config=KubernetesRun(),

A conceptual question.
So does the flow need to mention an image reference(from docker hub or private registry) as well as a storage reference ?
If there is an image reference being provided what is the need to provide the github storage path and access token?
A docker image should have everything that Kubernetes needs to run, right?

Ideally, yes. Explicit is better than implicit.

There is a separation of concerns.

  • Your Docker image set on the run_config is to package code and execution environment dependencies.
  • Your Storage configuration is for flow code only