Kubernetes Agent Job Logs

Are the job logs suppose to be collected by the agent? I’m experience some wierd behavior, sometimes the job output is shown at the agent stdout and other times it doesn’t. Am I missing some configuration?

21:24:53.797 | DEBUG   | prefect.infrastructure.kubernetes-job - Job 'garnet-sheepgljfw': Starting watch for pod start...
21:24:53.821 | INFO    | prefect.infrastructure.kubernetes-job - Job 'garnet-sheepgljfw': Pod has status 'Pending'.
21:24:55.947 | INFO    | prefect.infrastructure.kubernetes-job - Job 'garnet-sheepgljfw': Pod has status 'Running'.
21:24:58.103 | DEBUG   | prefect.agent - Checking for flow runs...
21:25:03.324 | DEBUG   | prefect.agent - Checking for flow runs...
21:25:06.512 | DEBUG   | prefect.infrastructure.kubernetes-job - Job 'garnet-sheepgljfw': Starting watch for job completion

Thank you.

1 Like

If you mean flow logs, yes. If you mean logs from a K8s pod or ECSTask or some external job, it might be that the logs are streamed only once the external job completes

For K8s specifically, I believe the latter is true. Feel free to open a feature request and we could explore how we can stream those logs to Prefect backend sooner