[Enhancement] Optimize correlated join row count estimation to avoid repeated Statistics.buildFrom

[stephen-shelby]

Why I'm doing:

Correlated join equality predicates are common in multi-join queries, and the current statistics estimation path applies auxiliary predicates by repeatedly rebuilding Statistics objects (multiple Statistics.buildFrom(...).build() per join). This adds unnecessary CPU overhead and object allocations in a hot optimizer path.

What I'm doing:

• Optimize correlated inner join statistics estimation by collapsing repeated auxiliary row-count updates into a single scaling using (coef^{k}), avoiding O(n²) Statistics.buildFrom(...) calls.
• Remove queue rotation / repeated rebuilding in the “choose driving predicate” loop while keeping the estimation semantics unchanged (auxiliary predicates only dampen row count, column stats unchanged).

Fixes #issue

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
  • This pr needs auto generate documentation
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 4.1
  • 4.0
  • 3.5
  • 3.4

---

Note

Speeds up correlated inner-join cardinality estimation without changing semantics.

• Replaces queue-based driving-predicate loop with single-pass selection; apply UNKNOWN_AUXILIARY_FILTER_COEFFICIENT^(k-1) to scale auxiliary predicates
• Adds fast path for single equality predicate and returns best driving stats with adjusted row count
• Removes estimateByEqOnPredicates/estimateByAuxiliaryPredicates and unused imports

^{Written by Cursor Bugbot for commit 34e8eda. This will update automatically on new commits. Configure here.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

[github-actions]

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

[FE Incremental Coverage Report]

✅ pass : 15 / 16 (93.75%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/sql/optimizer/statistics/StatisticsCalculator.java	15	16	93.75%	[1651]

[github-actions]

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[alvin-phoenix-ai]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no bugs!

[github-actions]

@Mergifyio backport branch-4.0

[github-actions]

@Mergifyio backport branch-3.5

[github-actions]

@Mergifyio backport branch-4.1

[mergify]

backport branch-4.0

✅ Backports have been created

Details

• #67823 [Enhancement] Optimize correlated join row count estimation to avoid repeated Statistics.buildFrom (backport #67773) has been created for branch branch-4.0

[mergify]

backport branch-3.5

✅ Backports have been created

Details

• #67824 [Enhancement] Optimize correlated join row count estimation to avoid repeated Statistics.buildFrom (backport #67773) has been created for branch branch-3.5

[mergify]

backport branch-4.1

❌ No backport have been created

Details

• Backport to branch branch-4.1 failed

GitHub error: Branch not found