Skip to content

fix(coverage): find actual tests through spec helper calls - #1128

Merged
1 commit merged into
tirth8205:stagingfrom
azizur100389:codex/fix-test-helper-coverage
Nov 6, 2026
Merged

1 commit merged into
tirth8205:stagingfrom
azizur100389:codex/fix-test-helper-coverage

Conversation

@azizur100389

@azizur100389 azizur100389 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

React spec helpers can create a resolved Test -> view -> Widget CALLS path without a Widget -> Test TESTED_BY edge. Both tests_for and the change report currently lose that evidence because test-file callers are excluded from the caller walk.

Follow bounded, resolved incoming CALLS through test helpers until an actual Test declaration is reached. Return the real test identities, retain all tests sharing a helper, and report the path as indirect reachability at the existing full untested risk. A helper's test flag or TESTED_BY edges alone are insufficient; ambiguous links, same-name helpers, call cycles and caller limits remain guarded.

Closes #1109. The interface/type-alias exemption already merged into staging is covered by the React integration regression.

Validation:

  • Reproduced 5 failing cases before the fix.
  • New regressions: 11 passed, including a real TSX component/spec build, 17 tests sharing a helper, direct plus indirect evidence, cycles, ambiguity and independent caller budgets.
  • Graph, transitive coverage, changes and declaration-gap suites: 219 passed, 1 skipped.
  • Ruff, mypy (--platform linux, matching CI), and Bandit pass.

GitHub CI on 8209995: 17 successful checks, including Python 3.10–3.13 (5,056 passed per version, 86.65–86.69% coverage), browser tests, MCP end-to-end tests on Linux/macOS/Windows, and native Windows resource/daemon tests. The manual upgrade-path check is skipped as configured. No merge conflicts with staging.

@github-actions

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.55 (MEDIUM) — 18 changed function(s)/class(es), 42 affected flow(s), 2 test gap(s) (0 with no tested caller found, 2 reached only through a caller)

Risk-scored changes

Risk Level Symbol Location Tested
0.55 medium code_review_graph/graph.py::GraphStore code_review_graph/graph.py:691 yes
0.55 medium code_review_graph/graph.py::GraphStore.get_transitive_tests code_review_graph/graph.py:1269 yes
0.45 medium code_review_graph/graph.py::GraphStore.get_caller_test_routes code_review_graph/graph.py:1559 indirect
0.40 medium code_review_graph/graph.py::GraphStore._get_test_helper_routes code_review_graph/graph.py:1481 indirect
0.15 low tests/test_test_helper_routes.py::node tests/test_test_helper_routes.py:20 (test)
0.15 low tests/test_test_helper_routes.py::call tests/test_test_helper_routes.py:26 (test)
0.15 low tests/test_test_helper_routes.py::chain tests/test_test_helper_routes.py:30 (test)
0.05 low tests/test_test_helper_routes.py::graph tests/test_test_helper_routes.py:14 (test)
0.05 low tests/test_test_helper_routes.py::test_gap_report_names_the_test_that_calls_the_helper tests/test_test_helper_routes.py:43 (test)
0.05 low tests/test_test_helper_routes.py::test_tests_for_finds_all_tests_that_call_the_same_helper tests/test_test_helper_routes.py:53 (test)

Affected execution flows

  • process — criticality 0.74, 538 node(s) across 14 file(s)
  • _run — criticality 0.74, 565 node(s) across 16 file(s)
  • build_or_update_graph_tool — criticality 0.73, 619 node(s) across 24 file(s)
  • refactor_tool — criticality 0.72, 159 node(s) across 12 file(s)
  • main — criticality 0.72, 926 node(s) across 40 file(s)
  • ...and 37 more affected flow(s)

Reached only through a caller

No test names these directly, but a tested caller reaches them along the call path shown. That is a path in the call graph, not a record of execution -- the caller's tests may never take this branch. Treat it as where to look, not as coverage: these count as gaps and are scored as untested.

  • code_review_graph/graph.py::GraphStore._get_test_helper_routes (code_review_graph/graph.py:1481) — via code_review_graph/graph.py::GraphStore.get_transitive_tests, 1 hop(s)
  • code_review_graph/graph.py::GraphStore.get_caller_test_routes (code_review_graph/graph.py:1559) — via code_review_graph/changes.py::analyze_changes, 1 hop(s)

Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Helper-only test reachability incorrectly lowers coverage risk and can remove untested symbols from review priorities; risk scoring must exclude helper-route results and gain a regression test.

1 open issue
What changed in this PR

This PR addresses #1109 by letting graph queries find actual tests that reach production symbols through test-file helpers, improving React component test-gap reporting.

Changes:

  • Adds bounded incoming CALLS traversal from symbols through helpers to Test nodes.
  • Integrates helper routes into test lookup and indirect gap classification.
  • Adds regressions for shared helpers, ambiguous links, cycles, traversal limits, and a real TSX build.
File Description
tests/​test_test_helper_routes.py Adds helper-route regressions and React integration coverage.
code_review_graph/​graph.py Resolves helper-to-test paths for test queries and caller-route reporting.

Review effort: Balanced

Comment on lines +1469 to +1471
for route in self._get_test_helper_routes(
input_qns, max_depth + 1, max_frontier, self.get_repo_root(),
).values():
@tirth8205 tirth8205 closed this pull request by merging all changes into tirth8205:staging in 0160eb0 Nov 6, 2026
gowrishacv pushed a commit to gowrishacv/code-review-graph that referenced this pull request Nov 6, 2026
…ymbol (review of tirth8205#1128)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gowrishacv pushed a commit to gowrishacv/code-review-graph that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants