rust_snuba is not compatible with Kafka v4

[alistairmaclean]

Environment

I'm installing Sentry self-hosted with an external Kafka 4 cluster. Note that, Kafka v3.9 is end-of-life.

rust_snuba is using rdkafka-sys 4.8.0+2.3.0 (which is librdkafka 2.3.0), which faces a protocol incompatibility issue during SASL/SCRAM auth with Kafka v4. More info here

Updating to rdkafka-sys 4.9.0+2.10.0 should fix the issue.

Please refer to this equivalent PR in getsentry/relay: getsentry/relay#5523

Steps to Reproduce

1. Run Kafka v4
2. Configure a Snuba rust-consumer to connect to Kafka v4 with SASL authentication (eapItemsConsumer)
3. Observe authentication failure error, despite credentials being correct.

Expected Result

Connection to Kafka successful.

Actual Result

%3|1767887318.290|FAIL|rdkafka#producer-2| [thrd:sasl_ssl://kafka.company.com:9093/bootstrap]: sasl_ssl://kafka.company.com:9093/bootstrap: SASL authentication error: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (after 121ms in state AUTH_REQ, 4 identical error(s) suppressed)
2026-01-08T15:48:38.290914Z ERROR rdkafka::client: librdkafka: Global error: Authentication (Local: Authentication failure): sasl_ssl://kafka.company.com:9093/bootstrap: SASL authentication error: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256 (after 121ms in state AUTH_REQ, 4 identical error(s) suppressed)
2026-01-08T15:48:38.291039Z  WARN rdkafka::producer::base_producer: Ignored event 'Error' on base producer poll

[linear]

EAP-354

[Dav1dde]

Looks like that may be blocked on Arroyo getsentry/arroyo#457 (comment)