Can I prevent Kubernetes agent from adding Prefect-specific labels to the flow run pods (Kubernetes jobs for the flow run)?

View in #prefect-community on Slack

Chris_Martinez @Chris_Martinez: Hi Good morning, is anyone familiar with dropping the following labels from a job/task in a kubernetes agent:

prefect.io/flow_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx                                                                                                                                                            
prefect.io/flow_run_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxx                                                                                                                                                         prefect.io/identifier=xxxxxxxx

Kevin_Kho @Kevin_Kho: Looking into this, but why are you trying to drop them?

Chris_Martinez @Chris_Martinez: looking to trim some labels or possibly even join them into a single label due to constraints from grafana allowing us a max amount of labels

Kevin_Kho @Kevin_Kho: The agent uses them prefect/agent.py at master · PrefectHQ/prefect · GitHub to manage jobs though so I don’t think you can remove them

Chris_Martinez @Chris_Martinez: I would be fine using that specific label if that is used for removal but the other two seem as though they could either be combined or removed.

Kevin_Kho @Kevin_Kho: So we surely need flow_run_id . I don’t know about flow_id. We could open an issue. You can’t do this at the moment because it’s hardcoded. You could remove flow_id from here and re-deploy your agent, but then you’d need to change your image of course.

Chris_Martinez @Chris_Martinez: Okay makes sense, I appreciate the help with this

Kevin_Kho @Kevin_Kho: you can chime in here:

we’ll see what the core team says.

Reason why those labels exist and are usually helpful

  • They allow an admin to map pods to a flow run that created them
  • They allow auditability - admins might want to force them… otherwise you can have flows that run with no connection back to Prefect, which makes it harder to tell when Prefect started a container and who started it