Skip to content

[CORE-13570] Deflake test_timequery_with_local_gc#28070

Merged
WillemKauf merged 2 commits intoredpanda-data:devfrom
WillemKauf:timequery_test_fixes
Oct 15, 2025
Merged

[CORE-13570] Deflake test_timequery_with_local_gc#28070
WillemKauf merged 2 commits intoredpanda-data:devfrom
WillemKauf:timequery_test_fixes

Conversation

@WillemKauf
Copy link
Copy Markdown
Contributor

@WillemKauf WillemKauf commented Oct 15, 2025

Fixes https://redpandadata.atlassian.net/issues/CORE-13570.

This test intended to disable local time-based retention, but did not consider the property retention.local.target.ms for a tiered storage enabled topic.

Previously, local retention would be enforced by broker time, but after PR #27383, the test became flakey due to the now "active" retention enforcement.

Set retention.local-target.ms=-1 to disable time-based retention for tiered storage enabled topics for this test.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.2.x
  • v25.1.x
  • v24.3.x

Release Notes

  • none

This test intended to disable local retention, but did not consider the
property `retention.local.target.ms` for a tiered storage enabled topic.

Previously, local retention would be enforced by broker time, but after PR
redpanda-data#27383, the test became flakey
due to the now "active" retention enforcement.

Set `retention.local-target.ms=-1` to disable time-based retention for
tiered storage enabled topics for this test.
The request `log_segment_size` in `test_timequery_with_local_gc` was
actually being ignored at start-up due to the bounded property checks,
manifesting as a `WARN` level log line.

Disable bounded property checks so the test can run as requested.
rpk_rr_cluster.create_topic(self.topic_name, config=conf)
except:
self.logger.warn(f"Failed to create a read-replica topic")
self.logger.warn("Failed to create a read-replica topic")
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ruff linter unnecessary f-string removal

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Deflake the test_timequery_with_local_gc by disabling local time-based retention for tiered-storage topics and allowing a smaller-than-minimum log_segment_size during the test.

  • Disable time-based local retention by setting retention.local.target.ms to -1 for the test topic
  • Set __REDPANDA_TEST_DISABLE_BOUNDED_PROPERTY_CHECKS before starting the cluster to permit sub-minimum log_segment_size
  • Minor cleanup of a log message string

"redpanda.remote.read": "true",
"redpanda.remote.write": "true",
"retention.local.target.bytes": local_retention,
# See comment about disabling `retention.ms` below.
Copy link

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline comment references disabling retention.ms, but the code is setting retention.local.target.ms. Please update the comment to reflect the local retention setting to avoid confusion. For example: 'Disable time-based local retention for tiered storage by setting retention.local.target.ms to -1.'

Suggested change
# See comment about disabling `retention.ms` below.
# Disable time-based local retention for tiered storage by setting retention.local.target.ms to -1.

Copilot uses AI. Check for mistakes.
Comment on lines 810 to +811
except:
self.logger.warn(f"Failed to create a read-replica topic")
self.logger.warn("Failed to create a read-replica topic")
Copy link

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid a bare except, which swallows KeyboardInterrupt/SystemExit and makes debugging harder. Catch Exception explicitly and log the exception details.

Copilot uses AI. Check for mistakes.
rpk_rr_cluster.create_topic(self.topic_name, config=conf)
except:
self.logger.warn(f"Failed to create a read-replica topic")
self.logger.warn("Failed to create a read-replica topic")
Copy link

Copilot AI Oct 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logger.warn is deprecated; use logger.warning instead. Consider including exc_info to capture the exception context.

Suggested change
self.logger.warn("Failed to create a read-replica topic")
self.logger.warning("Failed to create a read-replica topic", exc_info=True)

Copilot uses AI. Check for mistakes.
@vbotbuildovich
Copy link
Copy Markdown
Collaborator

CI test results

test results on build#74273
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
WriteCachingFailureInjectionTest test_unavoidable_data_loss null integration https://buildkite.com/redpanda/redpanda/builds/74273#0199e90a-cf1f-4854-9ebc-167164255dd2 FLAKY 14/21 upstream reliability is '90.86651053864169'. current run reliability is '66.66666666666666'. drift is 24.19984 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=WriteCachingFailureInjectionTest&test_method=test_unavoidable_data_loss

@WillemKauf
Copy link
Copy Markdown
Contributor Author

/ci-repeat 5
skip-redpanda-build
skip-units
dt-repeat=10
tests/rptest/tests/timequery_test.py::TimeQueryTest.test_timequery_with_local_gc

Comment on lines +364 to +366
self.redpanda.set_environment(
{"__REDPANDA_TEST_DISABLE_BOUNDED_PROPERTY_CHECKS": "ON"}
)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh the warning was on bootup, not from something later like alter topic config. i think the later will fail, but indeed this just seems to warn silently on bootup.

@WillemKauf WillemKauf merged commit a60224c into redpanda-data:dev Oct 15, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants