Skip to content

Enforce CRD CEL validation rules client-side in libcalico-go#11973

Merged
caseydavenport merged 13 commits intomasterfrom
casey-go-cel
Mar 17, 2026
Merged

Enforce CRD CEL validation rules client-side in libcalico-go#11973
caseydavenport merged 13 commits intomasterfrom
casey-go-cel

Conversation

@caseydavenport
Copy link
Member

Description

Right now, CEL validation rules (x-kubernetes-validations) on our CRDs are only enforced by the Kubernetes API server. In etcd mode there's no API server in the picture, so these validations are just silently skipped — which means you can create resources that violate the CEL rules without any error.

This PR adds a new package libcalico-go/lib/validator/cel/ that compiles the CEL rules from our embedded CRD schemas into Go-executable validators at init time, using the K8s apiextensions-apiserver CEL libraries. The pipeline is roughly:

  1. Load all embedded CRDs via config.AllCRDs()
  2. Convert v1.JSONSchemaProps → internal → structuralschema.Structural
  3. Compile CEL validators via celvalidation.NewValidator() (returns nil for CRDs with no CEL rules)
  4. On validation, convert typed objects to unstructured and run the compiled CEL programs

This is wired into the existing v3.Validate() function, so both Go struct validation and CEL validation run together and errors from both are collected into a single ErrorValidation. Currently Tier and ClusterNetworkPolicy are the only CRDs with CEL rules, but any future x-kubernetes-validations will automatically be picked up.

Note: This is a prototype / proof-of-concept. A few things that could be improved in follow-ups:

  • Update-time validation with oldSelf (the current Validate() signature doesn't take an old object)
  • MutatingAdmissionPolicy enforcement (the K8s admission machinery is too tightly coupled to reuse directly, so this would likely need a different approach)

Copilot AI review requested due to automatic review settings March 3, 2026 03:35
@caseydavenport caseydavenport added docs-not-required Docs not required for this change release-note-not-required Change has no user-facing impact labels Mar 3, 2026
@caseydavenport caseydavenport requested a review from a team as a code owner March 3, 2026 03:35
@marvin-tigera marvin-tigera added this to the Calico v3.32.0 milestone Mar 3, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds client-side enforcement of CRD x-kubernetes-validations (CEL) within libcalico-go, so that resources are validated consistently in both Kubernetes datastore mode (API server enforces CEL) and etcd mode (no API server).

Changes:

  • Integrates CEL validation into the existing libcalico-go/lib/validator/v3.Validate() path and aggregates CEL + struct validation errors into a single ErrorValidation.
  • Introduces a new libcalico-go/lib/validator/cel package that compiles CEL programs from embedded CRD schemas and evaluates them against objects converted to unstructured form.
  • Adds unit/integration tests covering Tier CEL validation and combined (struct + CEL) error reporting.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
libcalico-go/lib/validator/v3/validator.go Runs CEL validation alongside existing validator.v9 struct validation and merges errors.
libcalico-go/lib/validator/v3/cel_integration_test.go Verifies CEL is invoked through v3.Validate() and that errors are combined.
libcalico-go/lib/validator/cel/cel.go Loads embedded CRDs, builds structural schemas, compiles CEL validators once, and validates objects.
libcalico-go/lib/validator/cel/cel_test.go Tests CEL validator initialization and Tier CEL rule enforcement.

Add a new cel validator package that compiles x-kubernetes-validations
rules from embedded CRD schemas into Go-executable CEL programs at init
time, using the K8s apiextensions-apiserver CEL libraries.

Wire it into the existing v3 Validate() function so CEL rules are
enforced regardless of datastore backend. In Kubernetes mode the API
server already enforces these via CRDs, but in etcd mode there's no API
server — so without this, CEL validations were simply not checked.

Both Go struct validation errors and CEL errors are now collected and
returned together in a single ErrorValidation.
Copy link
Member

@mazdakn mazdakn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall seems good. Left a few comments and nits.

- Move CEL validation from separate cel package into validator/v3 as
  crd_validation.go
- Add OpenAPI schema validation alongside CEL (enforces enum, minItems,
  maxLength, pattern, etc. from CRD schemas)
- Find storage version instead of assuming Versions[0]
- Infer Kind from Go type when TypeMeta isn't set
- Use logrus directly instead of aliasing as log
- Consolidate var block, use init() instead of sync.Once
- Replace custom contains helpers with strings.Contains
- Restructure tests as per-Kind top-level tests with shared crdTestCase
  runner
@caseydavenport caseydavenport requested a review from a team as a code owner March 16, 2026 20:50
Copy link
Member

@mazdakn mazdakn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The CEL rules for tier defaultAction were crashing with "no such key"
when defaultAction wasn't set, because the field is omitempty. Add
has() guards so missing values are properly rejected instead of
causing a runtime error.

Also move CRD loading from the legacy libcalico-go/config/crd/ embed
to the canonical api/ module CRDs, which are the ones generated from
the Go type annotations and always have up-to-date CEL rules.

Update tier tests to set DefaultAction on well-known tiers and add
negative test cases for wrong default actions.
@marvin-tigera
Copy link
Contributor

Removing "merge-when-ready" label due to new commits

CRD schema validation now catches additional constraint violations
(enum, maxLength) alongside Go struct validation, producing multi-field
errors. The error format changes from "error with field X" (singular)
to a bulleted list, breaking substring assertions. Strip the prefix
so assertions match the field-specific content in both formats.
# Conflicts:
#	api/pkg/apis/projectcalico/v3/tier.go
- BGPPeer CEL: cast literal 0 to uint() for ASNumber (uint32) comparisons,
  fixing "expected int, got uint64" errors in CEL evaluation.
- IPAMConfig test: use ContainSubstring instead of Equal since the error now
  includes both Go validator and CRD schema validation messages.
- IP Pool controller test: add time.Sleep between pool creates to ensure
  distinct creation timestamps for deterministic sort ordering.
The CRD schema represents asNumber as int (not uint), so CEL
comparisons must use int literals. The uint(0) cast caused CEL
compilation failures.
Use JSON round-trip in toUnstructured() instead of
runtime.DefaultUnstructuredConverter, which preserves Go types via
reflection (e.g. uint32 → uint64). The API server receives JSON where
numbers are float64/int64, so the converter must match that. Normalize
float64 values to int64 where exact, fixing CEL runtime errors like
"expected int, got uint64" for fields like numorstring.ASNumber.

Add BGPPeer ASNumber CEL tests to cover this case.
@caseydavenport caseydavenport merged commit f64c2b6 into master Mar 17, 2026
2 of 3 checks passed
@caseydavenport caseydavenport deleted the casey-go-cel branch March 17, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required Docs not required for this change release-note-not-required Change has no user-facing impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants