Skip to content

iceberg: add iceberg_dlq_table_suffix configuration property#28242

Merged
nvartolomei merged 1 commit intoredpanda-data:devfrom
nvartolomei:nv/dlq-suffix
Oct 30, 2025
Merged

iceberg: add iceberg_dlq_table_suffix configuration property#28242
nvartolomei merged 1 commit intoredpanda-data:devfrom
nvartolomei:nv/dlq-suffix

Conversation

@nvartolomei
Copy link
Copy Markdown
Contributor

@nvartolomei nvartolomei commented Oct 28, 2025

iceberg: add iceberg_dlq_table_suffix configuration property

Introduces a new configuration property iceberg_dlq_table_suffix to make
the DLQ table suffix customizable. This is important for Iceberg catalogs
that do not support tilde (~) characters in table names. The default value
remains "~dlq" to maintain backward compatibility.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.2.x
  • v25.1.x
  • v24.3.x

Release Notes

Features

  • Iceberg: Add configurable DLQ table suffix to support catalogs that don't allow tilde characters and hence default <topic name>~dlq format.

@nvartolomei nvartolomei requested a review from a team as a code owner October 28, 2025 18:58
Copilot AI review requested due to automatic review settings October 28, 2025 18:58

This comment was marked as outdated.

oleiman
oleiman previously approved these changes Oct 28, 2025
Copy link
Copy Markdown
Member

@oleiman oleiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm


wait_until(lambda: count_rows() == count, timeout_sec=60, backoff_sec=1)

@gcp_only_test
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: this will be cdt-only, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Will trigger a run as soon as everything else passes.

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

Retry command for Build#75147

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/tls_version_test.py::TLSVersionTestRSA.test_change_version@{"version":1}

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

vbotbuildovich commented Oct 28, 2025

CI test results

test results on build#75147
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
LogCompactionTxRemovalTest test_tx_control_batch_removal null integration https://buildkite.com/redpanda/redpanda/builds/75147#019a2c67-1eae-441a-9928-72596195bfea FLAKY 10/21 upstream reliability is '93.38235294117648'. current run reliability is '47.61904761904761'. drift is 45.76331 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=LogCompactionTxRemovalTest&test_method=test_tx_control_batch_removal
NodesDecommissioningTest test_recommissioning_node {"cloud_topic": true} integration https://buildkite.com/redpanda/redpanda/builds/75147#019a2c70-2a58-4fbe-b6a9-5f9078accb45 FLAKY 20/21 upstream reliability is '98.91304347826086'. current run reliability is '95.23809523809523'. drift is 3.67495 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=NodesDecommissioningTest&test_method=test_recommissioning_node
ScalingUpTest test_fast_node_addition null integration https://buildkite.com/redpanda/redpanda/builds/75147#019a2c70-2a4d-4a44-9d96-012b95dcaeb7 FLAKY 18/21 upstream reliability is '99.58333333333333'. current run reliability is '85.71428571428571'. drift is 13.86905 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ScalingUpTest&test_method=test_fast_node_addition
TLSVersionTestRSA test_change_version {"version": 1} integration https://buildkite.com/redpanda/redpanda/builds/75147#019a2c70-2a53-4c56-bd09-cf529061db69 FAIL 0/1 https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=TLSVersionTestRSA&test_method=test_change_version
test results on build#75224
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
DeleteRecordsTest test_delete_records_concurrent_truncations {"cloud_storage_enabled": true, "truncate_point": "at_high_watermark"} integration https://buildkite.com/redpanda/redpanda/builds/75224#019a30ea-ce6e-46d8-a8e2-8df1070097c0 FLAKY 14/21 upstream reliability is '97.67441860465115'. current run reliability is '66.66666666666666'. drift is 31.00775 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DeleteRecordsTest&test_method=test_delete_records_concurrent_truncations
DeleteRecordsTest test_delete_records_concurrent_truncations {"cloud_storage_enabled": true, "truncate_point": "random_offset"} integration https://buildkite.com/redpanda/redpanda/builds/75224#019a30ea-ce67-4877-932b-b51b00e6459f FLAKY 12/21 upstream reliability is '93.84288747346072'. current run reliability is '57.14285714285714'. drift is 36.70003 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DeleteRecordsTest&test_method=test_delete_records_concurrent_truncations
PartitionMovementTest test_empty {"num_to_upgrade": 0} integration https://buildkite.com/redpanda/redpanda/builds/75224#019a30ea-ce67-4859-a45a-964495cc14b2 FLAKY 17/21 upstream reliability is '88.51963746223565'. current run reliability is '80.95238095238095'. drift is 7.56726 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=PartitionMovementTest&test_method=test_empty

andrwng
andrwng previously approved these changes Oct 28, 2025
@nvartolomei
Copy link
Copy Markdown
Contributor Author

/cdt
provider=gcp
region=us-west2
cdt_instance_type=n2-highmem-4
num_nodes=4
tests/rptest/tests/datalake/biglake_test.py

Introduces a new configuration property `iceberg_dlq_table_suffix` to make
the DLQ table suffix customizable. This is important for Iceberg catalogs
that do not support tilde (~) characters in table names. The default value
remains "~dlq" to maintain backward compatibility.
@nvartolomei
Copy link
Copy Markdown
Contributor Author

Force-push to fix mistakes in BigLake test.

  1. DuckDB isn't actually supported with GCP CDT.
  2. Forgot to enable schemaful translation.

@nvartolomei nvartolomei merged commit 550ae8c into redpanda-data:dev Oct 30, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants