[CORE-13202] config: introduce sasl mechanisms override by IoannisRP · Pull Request #27458 · redpanda-data/redpanda

IoannisRP · 2025-09-04T10:36:28Z

implements: CORE-13202

Backports Required

Release Notes

Features

Introduce new "sasl_mechanisms_override" configuration that allows per-listener controls over the sasl mechanisms.

src/v/config/configuration.cc

asimms41 · 2025-09-04T12:07:35Z

src/v/config/configuration.cc

+      "sasl_mechanisms_overrides",
+      "A list of sasl mechanisms overrides, defined by listener. Same "
+      "limitations "
+      "apply as in `sasl_mechanisms`.",


Suggested change

"apply as in `sasl_mechanisms`.",

"apply as for `sasl_mechanisms`.",

Should this also mention enterprise_sasl_mechanism?

Should this also mention enterprise_sasl_mechanism?

We don't even describe what is an enterprise value for the sasl_mechanisms 😅
Probably something to deal with in a follow up pr

asimms41

Just a few suggestions.

vbotbuildovich · 2025-09-04T15:00:09Z

CI test results

test results on build#71775

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason
RandomNodeOperationsTest	test_node_operations	{"cloud_storage_type": 1, "compaction_mode": "adjacent_merge", "enable_failures": true, "mixed_versions": true, "with_iceberg": false}	integration	https://buildkite.com/redpanda/redpanda/builds/71775#01991495-fa31-4597-8733-5f7005a76590	FLAKY	20/21	upstream reliability is '97.77070063694268'. current run reliability is '95.23809523809523'. drift is 2.53261 and the allowed drift is set to 50. The test should PASS
RandomNodeOperationsTest	test_node_operations	{"cloud_storage_type": 2, "compaction_mode": "chunked_sliding_window", "enable_failures": true, "mixed_versions": true, "with_iceberg": false}	integration	https://buildkite.com/redpanda/redpanda/builds/71775#01991495-fa32-4a24-988e-49275316b117	FLAKY	19/21	upstream reliability is '99.21875'. current run reliability is '90.47619047619048'. drift is 8.74256 and the allowed drift is set to 50. The test should PASS

test results on build#71804

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason
CloudStorageScrubberTest	test_scrubber	{"cloud_storage_type": 2}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#0199193d-0195-4b01-b829-46ce56a0fc4c	FLAKY	18/21	upstream reliability is '99.3975903614458'. current run reliability is '85.71428571428571'. drift is 13.6833 and the allowed drift is set to 50. The test should PASS
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "gzip"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#01991937-83fb-4d25-a296-0be94afbba92	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "gzip"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#0199193d-0195-4b01-b829-46ce56a0fc4c	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "lz4"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#01991937-83fc-4d3e-8179-edc8e2f7147e	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "lz4"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#0199193d-0196-45fa-88af-8c53c5b468c6	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "snappy"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#01991937-83fd-4bc8-bbb1-0b34f4a7773f	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "snappy"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#0199193d-0197-4140-81cf-d1689f21e590	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "zstd"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#01991937-83fd-4226-8c2e-6214a17ae885	FAIL	0/21	The test has failed across all retries
JavaCompressionTest	test_upgrade_java_compression	{"compression_type": "zstd"}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#0199193d-0197-49ac-bfd5-9a8e3b9d0bed	FAIL	0/21	The test has failed across all retries
RandomNodeOperationsTest	test_node_operations	{"cloud_storage_type": 2, "compaction_mode": "adjacent_merge", "enable_failures": true, "mixed_versions": true, "with_iceberg": false}	integration	https://buildkite.com/redpanda/redpanda/builds/71804#01991937-83fd-4bc8-bbb1-0b34f4a7773f	FLAKY	20/21	upstream reliability is '99.19028340080972'. current run reliability is '95.23809523809523'. drift is 3.95219 and the allowed drift is set to 50. The test should PASS

test results on build#71893

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason
DatalakeE2ETests	test_json_schema_unicode	{"catalog_type": "rest_hadoop", "cloud_storage_type": 1, "query_engine": "trino"}	integration	https://buildkite.com/redpanda/redpanda/builds/71893#01992dfb-36ff-43f1-96d0-bb9b77ad6711	FLAKY	20/21	upstream reliability is '99.64912280701755'. current run reliability is '95.23809523809523'. drift is 4.41103 and the allowed drift is set to 50. The test should PASS
RandomNodeOperationsTest	test_node_operations	{"cloud_storage_type": 2, "compaction_mode": "adjacent_merge", "enable_failures": true, "mixed_versions": true, "with_iceberg": false}	integration	https://buildkite.com/redpanda/redpanda/builds/71893#01992e00-073b-4aad-9703-2f7769805a6e	FLAKY	19/21	upstream reliability is '99.06976744186046'. current run reliability is '90.47619047619048'. drift is 8.59358 and the allowed drift is set to 50. The test should PASS

test results on build#72162

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason
PartitionBalancerTest	test_recovery_mode_rebalance_finish	null	integration	https://buildkite.com/redpanda/redpanda/builds/72162#01993eb1-1eb6-4e10-89d0-7dc60112966a	FLAKY	14/21	upstream reliability is '94.81865284974094'. current run reliability is '66.66666666666666'. drift is 28.15199 and the allowed drift is set to 50. The test should PASS
SimpleEndToEndTest	test_relaxed_acks	{"write_caching": false}	integration	https://buildkite.com/redpanda/redpanda/builds/72162#01993eb1-1eb3-43b2-91fe-1da449021db6	FLAKY	20/21	upstream reliability is '98.83720930232558'. current run reliability is '95.23809523809523'. drift is 3.59911 and the allowed drift is set to 50. The test should PASS

test results on build#72245

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason
SchemaEvolutionE2ETests	test_legal_schema_evolution	{"catalog_type": "rest_hadoop", "cloud_storage_type": 1, "produce_mode": "avro", "query_engine": "trino", "test_case": "add_column"}	integration	https://buildkite.com/redpanda/redpanda/builds/72245#01994ce8-4890-4d08-81f4-a55b79247d77	FLAKY	20/21	upstream reliability is '99.64285714285714'. current run reliability is '95.23809523809523'. drift is 4.40476 and the allowed drift is set to 50. The test should PASS
SchemaEvolutionE2ETests	test_old_schema_writer	{"catalog_type": "rest_hadoop", "cloud_storage_type": 1, "produce_mode": "avro", "query_engine": "trino", "test_case": "promote_column"}	integration	https://buildkite.com/redpanda/redpanda/builds/72245#01994ce8-4890-4d08-81f4-a55b79247d77	FLAKY	19/21	upstream reliability is '100.0'. current run reliability is '90.47619047619048'. drift is 9.52381 and the allowed drift is set to 50. The test should PASS

IoannisRP · 2025-09-05T09:15:28Z

Changes in force-push:

update config description

vbotbuildovich · 2025-09-05T10:46:06Z

Retry command for Build#71804

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"lz4"}
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"zstd"}
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"snappy"}
tests/rptest/tests/compatibility/java_compression_test.py::JavaCompressionTest.test_upgrade_java_compression@{"compression_type":"gzip"}

asimms41

LGTM

michael-redpanda

Nice!

Can we maybe expand that PLAIN test to validate that if PLAIN is in the global list but not in the overrides that client attempting to connect via PLAIN fails?

michael-redpanda · 2025-09-08T14:32:46Z

src/v/config/sasl_mechanisms.h

+inline constexpr auto enterprise_sasl_mechanisms
+  = std::to_array<std::string_view>({gssapi, oauthbearer});
+
+bool enterprise_sasl_mechanism(const ss::sstring& sasl_mech);


two nits:

docs

maybe should be is_enterprise_sasl_mechanism

michael-redpanda · 2025-09-08T14:34:07Z

src/v/config/sasl_mechanisms.h

+bool enterprise_sasl_mechanism(const ss::sstring& sasl_mech);
+
+// Checks if `sasl_mech` is enabled in sasl_mechanisms config
+bool has_sasl_mechanism(const std::string_view sasl_mech);


suggestion: to make this easier to use in tests, maybe also supply a reference to the config store you wish to check against rather than always using config::shard_local_cfg()

config_store deals in base_property which doesn't know anything about the concrete types of the property values. We would have to serialize the property (which is the only way to inspect a property value through a base_property) and do string manipulations to see if sasl_mech exists or not in it.

IoannisRP · 2025-09-09T09:18:44Z

changes in force-push:

update config/sasl_mechanisms.h api (naming + comments)
extend PLAIN sasl dt test to check disabling PLAIN through an override

dotnwat · 2025-09-09T17:52:11Z

src/v/config/sasl_mechanisms.h

+bool is_enterprise_sasl_mechanism(const ss::sstring& sasl_mech);
+
+// Checks if `sasl_mech` is enabled in sasl_mechanisms config
+bool has_sasl_mechanism(const std::string_view sasl_mech);


why const? string_view is already intended to be read-only.

same difference between a const ptr* and const ptr* const.

It is not about the underlying data that the string_view is pointing at. It is about the string_view itself.
I am a big fan of "const-everything", in general, even when it doesn't make a significant (any?) difference.

Following the same logic, i tend to prefer void foo(const int) over void foo(int) (assuming the value doesn't change during the function execution).

pgellert

lgtm

src/v/security/oidc_service.cc

pgellert · 2025-09-10T14:09:54Z

src/v/config/sasl_mechanisms.cc

+    const auto overrides = config::shard_local_cfg().sasl_mechanisms_overrides()
+                           | std::views::transform(
+                             &sasl_mechanisms_override::sasl_mechanisms);


Do we need to check here if a particular listener exists at all? I think it's fine not to, but wondering if others have an opinion here.

src/v/config/configuration.cc

IoannisRP · 2025-09-12T15:05:31Z

changes in force-push

rebase to dev (includes security report feature)

changes in force-push:

implement changes to security report to include the override behavior

This new config is a way to control the cluster's sasls_mechanisms on a per-listener basis. For a given kafka listener, if there is an entry in the sasl_mechanisms_overrides config, the value from sasl_mechanisms will be ignored and the override will be used in its place. For any functionality that needs to know if a sasl mechanisms is active, like for the enterprise features check, it is considered active if it is used in any listener override.

IoannisRP · 2025-09-15T09:29:11Z

changes in force-push:

update config descriptions
change config visibility from tunable to user

pgellert

lgtm

src/v/redpanda/admin/security.cc

IoannisRP self-assigned this Sep 4, 2025

github-actions bot added area/build area/redpanda labels Sep 4, 2025

IoannisRP force-pushed the CORE-13202/sasl-mechanisms-override branch from c23baa3 to 826d0c2 Compare September 4, 2025 10:48

IoannisRP marked this pull request as ready for review September 4, 2025 10:50

IoannisRP requested a review from a team as a code owner September 4, 2025 10:50

IoannisRP requested review from a team, michael-redpanda and pgellert and removed request for a team September 4, 2025 10:50

asimms41 reviewed Sep 4, 2025

View reviewed changes

src/v/config/configuration.cc Outdated Show resolved Hide resolved

asimms41 reviewed Sep 4, 2025

View reviewed changes

IoannisRP force-pushed the CORE-13202/sasl-mechanisms-override branch from 826d0c2 to 6aa412a Compare September 5, 2025 08:59

IoannisRP requested a review from asimms41 September 8, 2025 09:44

asimms41 previously approved these changes Sep 8, 2025

View reviewed changes

michael-redpanda reviewed Sep 8, 2025

View reviewed changes

IoannisRP dismissed asimms41’s stale review via f8a75db September 9, 2025 09:17

IoannisRP force-pushed the CORE-13202/sasl-mechanisms-override branch from 6aa412a to f8a75db Compare September 9, 2025 09:17

IoannisRP requested a review from michael-redpanda September 9, 2025 09:37

dotnwat reviewed Sep 9, 2025

View reviewed changes

pgellert previously approved these changes Sep 10, 2025

View reviewed changes

pgellert reviewed Sep 10, 2025

View reviewed changes

src/v/config/configuration.cc Outdated Show resolved Hide resolved

config: introduce encapsulation for sasl_mechanisms logic

0142163

IoannisRP dismissed pgellert’s stale review via 014229b September 12, 2025 11:07

IoannisRP force-pushed the CORE-13202/sasl-mechanisms-override branch from f8a75db to 014229b Compare September 12, 2025 11:07

IoannisRP force-pushed the CORE-13202/sasl-mechanisms-override branch from 014229b to 9ed3043 Compare September 12, 2025 15:04

IoannisRP force-pushed the CORE-13202/sasl-mechanisms-override branch from 9ed3043 to 5a79084 Compare September 15, 2025 09:23

IoannisRP requested review from dotnwat and pgellert September 15, 2025 09:29

pgellert approved these changes Sep 15, 2025

View reviewed changes

src/v/redpanda/admin/security.cc Show resolved Hide resolved

michael-redpanda approved these changes Sep 16, 2025

View reviewed changes

IoannisRP merged commit bb7f9af into redpanda-data:dev Sep 16, 2025
18 checks passed

IoannisRP mentioned this pull request Oct 1, 2025

[v25.2.x][CORE-13202] config: introduce sasl mechanisms override #27833

Merged

7 tasks

	"apply as in `sasl_mechanisms`.",
	"apply as for `sasl_mechanisms`.",

Conversation

IoannisRP commented Sep 4, 2025 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backports Required

Release Notes

Features

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asimms41 left a comment

Choose a reason for hiding this comment

Uh oh!

vbotbuildovich commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI test results

Uh oh!

IoannisRP commented Sep 5, 2025

Uh oh!

vbotbuildovich commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Retry command for Build#71804

Uh oh!

asimms41 left a comment

Choose a reason for hiding this comment

Uh oh!

michael-redpanda left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

IoannisRP commented Sep 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pgellert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

IoannisRP commented Sep 12, 2025

Uh oh!

IoannisRP commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pgellert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

IoannisRP commented Sep 4, 2025 •

edited by atlassian bot

Loading

vbotbuildovich commented Sep 4, 2025 •

edited

Loading

vbotbuildovich commented Sep 5, 2025 •

edited

Loading

IoannisRP commented Sep 15, 2025 •

edited

Loading