Securing and reducing privilages to kargo-management-controller

[scott-hpe]

It looks like the cluster role kargo-management-controller has wildcard privilages on clusterroles, clusterrolebindings, rolebindings, and roles. Can this scope safely be reduced to remove the clusterroles and clusterrolebindings?

Are there specific recommendations to secure this highly privilaged cluster role to prevent privilage escalation?

kargo-managment-controller cluster role

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    meta.helm.sh/release-name: kargo
    meta.helm.sh/release-namespace: kargo
  creationTimestamp: "2026-03-08T02:33:24Z"
  labels:
    app.kubernetes.io/component: management-controller
    app.kubernetes.io/instance: kargo
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kargo
    app.kubernetes.io/version: v1.9.5
    helm.sh/chart: kargo-1.9.5
  name: kargo-management-controller
  resourceVersion: "1086"
  uid: a4076f26-70a7-4066-beca-0256dc98e166
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - serviceaccounts
  verbs:
  - '*'
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterroles
  - clusterrolebindings
  - rolebindings
  - roles
  verbs:
  - '*'
- apiGroups:
  - kargo.akuity.io
  resources:
  - projects
  verbs:
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - kargo.akuity.io
  resources:
  - clusterconfigs/status
  - projects/status
  - projectconfigs/status
  verbs:
  - patch
- apiGroups:
  - kargo.akuity.io
  resources:
  - projectconfigs
  verbs:
  - create
- apiGroups:
  - kargo.akuity.io
  resources:
  - clusterconfigs
  - stages
  - warehouses
  - projectconfigs
  verbs:
  - get
  - list
  - watch

[krancour]

It needs those privileges, and the whole reason it has them is so that other components don't.

It would be far, far more dangerous for the API server or for a (non-management) controller to have such broad access.

The role of the management controller is to grant broad access only to a single component in the control plane and then allow that component to dynamically expand and contract much narrower sets of permissions for all the other components that are inherently more vulnerable.

[scott-hpe]

I believe you’re suggesting that the management-controller itself is a hardened secure resource within the cargo control plane. What makes it particularly secure or how is it hardened? Could it also be considered a high-value attack vector that, if compromised, could be used to grant additional privileges to other components?

I’m trying to understand this better because the diagram provided in the docs.kargo.io/operator-guide/architecture/#the-kargo-control-plane doesn’t provide enough detail about the attack vectors for the management-controller.

[krancour]

From the same docs you linked to:

Because some components are internet-facing (the API server and external webhooks Server) and others (controllers) can be distributed to other clusters, operators are often reluctant to provide these components with broad access to sensitive resources, particularly Kubernetes Secrets in the control plane. To mitigate this, the management controller, which is neither internet-facing nor distributed, is granted broad permissions and dynamically expands and contracts narrower sets of permissions for other components and Kargo users as Projects are created and deleted.

[scott-hpe]

I get the concept of what you are saying, broad privilages on the managment controller, which is ideally isolated, and principle of least privilaged (PLP) being applied dynamically to other components in the control plane. This point is well taken, but also the comment about secrets stands out to me:

operators are often reluctant to provide these components with broad access to sensitive resources, particularly Kubernetes Secrets in the control plane

The kargo-webhooks-server has unrestricted access to secrets across the entire cluster. Could this access be restricted to only the namespaces where Kargo projects reside?

When I look at the kargo-webhooks-server having unrestricted access to secrets, configmaps, serviceaccounts, roles, and rolebindings, I’m concerned that it may not be adhering to PLP. Shouldn’t the management controller grant access to the kargo-webhooks-server privileges as required to read secrets from project namespaces as the docs state?

[krancour]

I can see where that would raise a red flag. It may be an unfortunate accident of naming. kargo-webhooks-server serves Kubernetes dynamic admission controls. Its only client is the Kubernetes API server. It is never internet-facing.

This component existed long-before the need to handle webhooks from external systems arose, so when we started doing so, the desirable name "webhooks server" was taken and the new component was christened the "external webhooks server."

As to why it has some permissions that may seem unnecessarily broad, its responsibilities do overlap with the management controller somewhat. The management controller is great at dynamically expanding and contracting the permissions of other components as Projects are created and deleted, but it does so asynchronously, as controllers do. There are some cases where some of that needs to happen synchronously.

[scott-hpe]

For the kargo-management-controller, what operations would need the create of new ClusterRoles and ClusterRoleBindings for a Project scope? I understand your arguments regarding the reduced attack surface due to the lack of internet exposure, but the platform team sees it a highly privileged resource with a potential path to privilege escalation.

As a GitOps-driven environment utilizing ESO and vending our own Kargo projects and namespaces, could we also provide the necessary permission scope changes to restrict the default permissions of the kargo-webhooks-server from being cluster-wide? Same for can we reduce the permissions around ClusterRoles and ClusterRoleBindings for the kargo-managment-controller?

[krancour]

For the kargo-management-controller, what operations would need the create of new ClusterRoles and ClusterRoleBindings for a Project scope?

It's to manage permissions on Project resources, which are themselves, cluster-scoped.

kargo/pkg/controller/management/projects/projects.go

Lines 1085 to 1087 in b77e75d

	// This ClusterRole allows those bound to it to update and delete one specific
	// Project. This is necessary since Projects are cluster-scoped, meaning this
	// permission cannot be defined anywhere except at the cluster-level.

[scott-hpe]

As an aside, when I was reviewing roles and cluster roles for the platform team here, I noticed in kargo-webhooks-server that for kargo.akuity.io it listed project/status instead of projects/status as listed elsewhere. Is this a typo?

kargo/charts/kargo/templates/kubernetes-webhooks-server/cluster-role.yaml

Line 60 in b77e75d

- project/status