[CORE-12502] Add NFTablesSupport setting (default Enabled)#12146
[CORE-12502] Add NFTablesSupport setting (default Enabled)#12146nelljerram wants to merge 1 commit intoprojectcalico:masterfrom
Conversation
So that we have an escape hatch if nftables is not available at all on someone's distribution, or if their kernel errors when nftables is used. Note that nftables _support_ is different from nftables _mode_. The latter configures if we use nftables for policy programming; the former configures if we can use nftables for other purposes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
marvin-tigera
added
release-note-required
There was a problem hiding this comment.
Pull request overview
Adds a new FelixConfiguration knob (nftablesSupport, default Enabled) intended as an escape hatch to disable Felix’s non-policy use of nftables (notably the ARP-family table used for proxy-ARP suppression), while clarifying the meaning of existing nftablesMode.
Changes:
- Add
nftablesSupportto FelixConfiguration API/CRD schemas and generated manifests/docs. - Wire
NFTablesSupportthrough Felix config into the internal dataplane, gating creation of the ARP-family nftables table. - Update wording for
NFTablesModeto clarify it is specifically about policy programming.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| manifests/v3_projectcalico_org.yaml | Adds nftablesSupport to v3 schema output. |
| manifests/v1_crd_projectcalico_org.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/operator-crds.yaml | Updates nftablesMode description; adds nftablesSupport schema for operator bundle. |
| manifests/flannel-migration/calico.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/crds.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/canal.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/calico.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/calico-vxlan.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/calico-typha.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/calico-policy-only.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| manifests/calico-bpf.yaml | Updates nftablesMode description; adds nftablesSupport schema. |
| libcalico-go/config/crd/crd.projectcalico.org_felixconfigurations.yaml | Updates nftablesMode description; adds nftablesSupport to libcalico-go CRD. |
| felix/docs/config-params.md | Updates NFTablesMode wording; documents new NFTablesSupport. |
| felix/docs/config-params.json | Updates generated config docs for NFTablesMode; adds NFTablesSupport metadata. |
| felix/dataplane/linux/int_dataplane_test.go | Adds a constructor test case for NFTablesSupport=Disabled. |
| felix/dataplane/linux/int_dataplane.go | Gates ARP-family nftables table creation on NFTablesSupport. |
| felix/dataplane/driver.go | Wires configParams.NFTablesSupport into internal dataplane config. |
| felix/config/config_params.go | Adds Felix config param NFTablesSupport. |
| api/pkg/openapi/generated.openapi.go | Updates OpenAPI description for nftablesMode; adds schema for nftablesSupport. |
| api/pkg/apis/projectcalico/v3/zz_generated.deepcopy.go | Adds deepcopy support for NFTablesSupport field. |
| api/pkg/apis/projectcalico/v3/felixconfig.go | Adds NFTablesSupport type and FelixConfigurationSpec field; updates NFTablesMode comment. |
| api/config/crd/projectcalico.org_felixconfigurations.yaml | Adds nftablesSupport to API CRD schema output. |
| // nftables configuration. | ||
| NFTablesMode string `config:"oneof(Enabled,Disabled,Auto);Auto"` | ||
| NFTablesMode string `config:"oneof(Enabled,Disabled,Auto);Auto"` | ||
| NFTablesSupport string `config:"oneof(Enabled,Disabled);Enabled"` |
| // NFTablesSupport is the enum used to enable/disable nftables support. | ||
| // +enum | ||
| // +kubebuilder:validation:Enum=Disabled;Enabled;Auto | ||
| type NFTablesSupport string | ||
|
|
||
| const ( | ||
| NFTablesSupportEnabled NFTablesSupport = "Enabled" | ||
| NFTablesSupportDisabled NFTablesSupport = "Disabled" | ||
| ) |
| "nftablesSupport": { | ||
| SchemaProps: spec.SchemaProps{ | ||
| Description: "NFTablesSupport configures whether nftables can be used for purposes other than policy programming in Felix. [Default: Enabled]\n\nPossible enum values:\n - `\"Disabled\"`\n - `\"Enabled\"`", | ||
| Type: []string{"string"}, | ||
| Format: "", | ||
| Enum: []interface{}{"Disabled", "Enabled"}, | ||
| }, |
| // we use nftables for policy programming; the former configures if we can use nftables for | ||
| // other purposes. | ||
| var arpRootTable *nftables.NftablesTable | ||
| if config.NFTablesSupport == string(apiv3.NFTablesSupportEnabled) { |
| Expect(requestedFamilies).NotTo(ContainElement(knftables.ARPFamily)) | ||
| }) | ||
| }) | ||
|
|
|
@nelljerram perhaps you could expand on why we would want nftables for one purpose but not the other? I'm a bit confused by this one (and expect the similarly named config options are going to be really confusing for users - we already get a lot of head scratching with IptablesBackend vs NftablesMode!) |
In #12038 I have used nftables with the "arp" family to suppress Calico sending a proxy ARP response to a migrating VM for its own IP. This use of nftables is independent of whether we use nftables/iptables/eBPF for policy programming. My understanding is that there are still scaling concerns with nftables for policy in some setups, so we didn't want to say "if you want to do that, you must also use nftables for policy"; hence the independence point. But then @fasaxc said: what if someone is using a distro or kernel where nftables use isn't available or causes a problem - we should have a way to disable the ARP suppression use if necessary. And that more or less works for the function concerned here - live migration proxy ARP suppression - because it's only needed in quite specific setups, and then we're no worse off than we were before all the live migration work. Hence I've added this new setting to allow us to do that. Hopefully that explains the situation and the motivation. I'm certainly happy to change the naming and commenting to make everything clearer. |
|
@nelljerram thanks - that makes more sense to me now! I might still just have an IIUC, this sounds like a pretty niche case - we expect this to work regardless of dataplane in most cases. |
So that we have an escape hatch if nftables is not available at all on someone's distribution, or if their kernel errors when nftables is used.
Note that nftables support is different from nftables mode. The latter configures if we use nftables for policy programming; the former configures if we can use nftables for other purposes.
Release note: