Skip to content

fix: name Laravel container tests in tests_for as guesses (part of #850) - #1176

Merged
2 commits merged into
stagingfrom
fix/850-laravel-tests-for
Nov 9, 2026
Merged

2 commits merged into
stagingfrom
fix/850-laravel-tests-for

Conversation

@tirth8205

Copy link
Copy Markdown
Owner

Part of #850. This is the cheap half the triage named: migration up/down, and a label so tests_for says when a test link is a guess. Resolving Laravel container calls properly is #851 and is not done here.

What was wrong

Laravel tests reach a class through the container, so the file never says which class a call runs on:

$this->service = app(IPinfoService::class);   // in setUp
$this->service->getCountryCodeFromIp('1.2.3.4');  // in the test

Two things went wrong, both seen on koel/koel:

  1. A false self-call. A PHP call on an expression ($this->service->x(), app(X::class)->x()) has no named receiver, so the parser bound it to a method of the same name in the calling file. PHPUnit tests are often named after the method they test, so the test was recorded as calling itself. koel had 113 such edges from tests. App code has the same bug, for example a controller invite() calling $this->invitationService->invite(). Python already marks these calls as receiver_form: expression; PHP now does too.
  2. No answer for an ambiguous call. When more than one class the test imports defines the method (the issue's $completion->execute()), the call stays unresolved with its candidates in ambiguous_targets, and tests_for gave 0. Triage saw exactly this: tests_for execute = 0 with bare TESTED_BY edges.

What changed

  • parser.py: PHP member calls on an expression get receiver_form: "expression" (one line).
  • graph.py: new GraphStore.get_ambiguous_call_tests(), PHP only. A test whose ambiguous call lists this method is returned, unless the test builds one of the other candidates itself ((new UpdateBooks)->handle() is taken to mean UpdateBooks).
  • tools/query.py: tests_for lists those tests with inferred_by: "ambiguous_call". The summary says how many rows are guesses, and minimal mode now keeps inferred_by on tests_for (it dropped the existing naming_convention label too).
  • changes.py: a guess is never coverage, so gaps and risk do not change; the summary adds one line when PHP gaps remain. up/down are not gaps in a migrations or migrate folder inside the repository (read from the repo-relative path, as test folders are).

Easy to merge with #1154: git merge-tree of this branch and #1154's head is clean. The new tests_for block sits above the naming-convention block #1154 edits, and the CHANGELOG entry is at the end of Fixed.

Proof

Shallow clones of koel/koel (2,809 files, 475 test files) and monicahq/monica (1,938 files, 516 test files), plus laravel/laravel, built with origin/staging and with this branch, each into its own CRG_DATA_DIR. PYTHONHASHSEED=0 for every query (see the note at the end).

tests_for, before → after, checked against the test source:

Class (koel unless noted) Before After Checked against the test source
app/Services/Geolocation/IPinfoService 0 2, both guesses setUp does app(IPinfoService::class); both tests call $this->service->getCountryCodeFromIp(). Both right. The first test is itself named getCountryCodeFromIp, so before, its call was bound to itself.
app/Services/Image/SvgSanitizer 0 1 guess sanitize calls $this->service->sanitize() on new SvgSanitizer(...). Right.
app/Services/Integrations/TicketmasterService 0 4 guesses All 4 tests call $this->service->searchEventForArtist() on app(TicketmasterService::class). All right.
app/Services/ScrobbleService 0 5 guesses All 5 tests call scrobble, updateNowPlaying or hasConnectedScrobbler on $this->service = new ScrobbleService(...). All right.
app/Services/Integrations/ListenBrainzService 1 14, 13 guesses 10 guesses are ListenBrainzServiceTest tests calling $this->service->... on app(ListenBrainzService::class): right. 3 are ScrobbleServiceTest tests, where ListenBrainzService is a Mockery mock: wrong, which is why they say they are guesses.
monica app/Domains/Contact/DavClient/Jobs/SynchronizeAddressBooks (job) 1 1 UpdateAddressBooksTest runs (new UpdateAddressBooks)->handle() under Queue::fake(), so this job's handle never runs. Before the "builds one of the classes" rule this gave 4 wrong guesses; with it, none.

The issue's own shape ($completion = app(CompleteSupplierCount::class); $completion->execute(1), with a second imported class that also has execute), in a small repo: tests_for app/Actions/CompleteSupplierCount.php gave 0 before and 1 guess after (test_it_completes, inferred_by: ambiguous_call).

Survey of every class under app/Services and app/Jobs (koel) and app/Domains, app/Services, app/Actions (monica):

koel before koel after monica before monica after
Classes surveyed 171 171 595 595
tests_for gives 0 42 37 49 49
tests_for gives 0 though a test file names the class 17 12 5 5
Classes whose answer gained guess rows 39 3
Classes whose traced (non-guess) rows changed 0 0
PHP CALLS edges from a test to itself 113 15 0 0

The 15 self-calls left on koel come from the post-build resolver, when the test's own method is the only candidate the file backs; that is separate from this change.

detect_changes on two koel migrations plus IPinfoService.php, before → after:

### old
Analyzed 3 changed file(s):
  - 10 changed function(s)/class(es)
  - 1 affected flow(s)
  - 9 test gap(s)
  - Overall risk score: 0.60
  - Untested: CreateUsersTable, up, down, CreateArtistsTable, IPinfoService
gaps: ['CreateArtistsTable', 'CreateUsersTable', 'IPinfoService', 'down', 'down', 'getCountryCodeFromIp', 'up', 'up', 'used']
### new
Analyzed 3 changed file(s):
  - 10 changed function(s)/class(es)
  - 1 affected flow(s)
  - 5 test gap(s)
  - Overall risk score: 0.60
  - Untested: CreateUsersTable, CreateArtistsTable, IPinfoService, used, getCountryCodeFromIp
  - PHP gaps are a guess: tests that reach code through the container or an HTTP call are not traced (tests_for names likely ones)
gaps: ['CreateArtistsTable', 'CreateUsersTable', 'IPinfoService', 'getCountryCodeFromIp', 'used']

Tests

  • tests/test_php_guessed_test_links.py (new, 10 tests): the self-call, the guess for a method, a class and a file, the label in minimal mode, a pinned call without a label, no guess without an import, no guess when the test builds another candidate, guesses not counted as coverage, PHP only.
  • tests/test_test_gap_exemptions.py: 4 migration tests, including a checkout under a migrations folder.
  • The tests that pin new behaviour fail on staging. The ones that guard against over-reach (no guess without an import, no label on a pinned call, no guess when the test builds another candidate, up outside a migration folder) pass on both.

Checks

  • uv run ruff check code_review_graph/: clean.
  • uv run --extra dev python -m mypy code_review_graph/ --ignore-missing-imports --no-strict-optional: no issues in 85 files.
  • uv run pytest tests/: all pass. The machine was under heavy load from other runs, so the suite ran in parts; one timing test, test_cpp_header_prefilter.py::test_text_check_stays_fast_on_hostile_input[enum], failed once under that load and passes when run alone (39 of 39 in that file).

Left for #851

Calls through X::dispatch(), $this->post(...) and constructor injection are still not traced; a test reached only that way still shows no link. detect_changes still counts these as gaps, now with a line saying PHP gaps are a guess.

Found on the way, not fixed here

tests_for on a large class is not stable between runs: get_transitive_tests cuts each hop to 50 callees taken from a Python set, so which callees survive depends on string hashing. On koel's PodcastService.php, three runs of the unchanged staging code gave 58, 51 and 19 results.

🤖 Generated with Claude Code

A PHP call on an expression, such as $this->service->run(), was bound to a
same-named method in the calling file, so a test named after the method it
tests was recorded as calling itself. Mark it as an expression call, as
Python already does.

When several imported classes define the called method, tests_for now lists
the test with inferred_by: ambiguous_call, unless the test builds one of
those classes itself. The summary counts the guesses, minimal mode keeps
inferred_by, and the test-gap summary says PHP gaps are a guess. Guesses
never count as coverage.

Migration up and down in a migrations or migrate folder are not test gaps.

Part of #850.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.85 (CRITICAL) — 37 changed function(s)/class(es), 47 affected flow(s), 6 test gap(s) (1 with no tested caller found, 5 reached only through a caller)

Risk-scored changes

Risk Level Symbol Location Tested
0.85 critical code_review_graph/tools/query.py::add_result code_review_graph/tools/query.py:550 indirect
0.75 high code_review_graph/changes.py::is_test_gap_exempt code_review_graph/changes.py:58 indirect
0.75 high code_review_graph/tools/query.py::query_graph code_review_graph/tools/query.py:488 yes
0.55 medium code_review_graph/changes.py::analyze_changes code_review_graph/changes.py:722 yes
0.55 medium code_review_graph/parser.py::CodeParser code_review_graph/parser.py:3205 yes
0.55 medium code_review_graph/tools/review.py::get_review_context code_review_graph/tools/review.py:428 yes
0.55 medium code_review_graph/tools/review.py::_generate_review_guidance code_review_graph/tools/review.py:753 yes
0.40 medium code_review_graph/changes.py::_is_migration_file code_review_graph/changes.py:46 indirect
0.40 medium code_review_graph/graph.py::GraphStore.owner code_review_graph/graph.py:1613 indirect
0.40 medium code_review_graph/parser.py::CodeParser._extract_calls code_review_graph/parser.py:15129 no

Affected execution flows

  • process — criticality 0.74, 655 node(s) across 18 file(s)
  • _run — criticality 0.74, 688 node(s) across 20 file(s)
  • build_or_update_graph_tool — criticality 0.73, 752 node(s) across 27 file(s)
  • refactor_tool — criticality 0.72, 165 node(s) across 12 file(s)
  • main — criticality 0.72, 1098 node(s) across 45 file(s)
  • ...and 42 more affected flow(s)

Test gaps

  • code_review_graph/parser.py::CodeParser._extract_calls (code_review_graph/parser.py:15129)

Reached only through a caller

No test names these directly, but a tested caller reaches them along the call path shown. That is a path in the call graph, not a record of execution -- the caller's tests may never take this branch. Treat it as where to look, not as coverage: these count as gaps and are scored as untested.

  • code_review_graph/changes.py::_is_migration_file (code_review_graph/changes.py:46) — via code_review_graph/changes.py::analyze_changes, 2 hop(s)
  • code_review_graph/changes.py::is_test_gap_exempt (code_review_graph/changes.py:58) — via code_review_graph/changes.py::analyze_changes, 1 hop(s)
  • code_review_graph/graph.py::GraphStore.owner (code_review_graph/graph.py:1613) — via code_review_graph/graph.py::GraphStore.get_ambiguous_call_tests, 1 hop(s)
  • code_review_graph/graph.py::GraphStore.built (code_review_graph/graph.py:1617) — via code_review_graph/graph.py::GraphStore.get_ambiguous_call_tests, 1 hop(s)
  • code_review_graph/tools/query.py::add_result (code_review_graph/tools/query.py:550) — via code_review_graph/tools/query.py::query_graph, 1 hop(s)

Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The test-wide constructor exclusion can incorrectly skip unrelated ambiguous calls, causing tests_for to return zero despite a relevant test.

1 open issue
What changed in this PR

This PR partially addresses #850 by labeling uncertain PHP test links and reducing migration-related test-gap noise. Full Laravel container resolution remains deferred to #851.

Changes:

  • Prevents PHP expression-receiver calls from binding to same-named local methods.
  • Adds labeled guesses to tests_for, preserving labels in minimal output without counting guesses as coverage.
  • Exempts migration up/down methods from test gaps and adds a PHP uncertainty notice.
File Description
tests/​test_test_gap_exemptions.py Tests migration exemptions and repository path boundaries.
tests/​test_php_guessed_test_links.py Tests PHP guessed links, labels, exclusions, and coverage separation.
docs/​FAQ.md Explains guessed links versus coverage.
docs/​COMMANDS.md Documents inference labels in tests_for output.
code_review_graph/​tools/​review.py Passes repository roots into gap-exemption checks.
code_review_graph/​tools/​query.py Adds guessed tests, summary counts, and minimal-mode labels.
code_review_graph/​parser.py Marks PHP expression receivers to avoid incorrect local binding.
code_review_graph/​graph.py Looks up PHP tests with ambiguous call candidates.
code_review_graph/​changes.py Adds migration exemptions and PHP gap uncertainty text.
CHANGELOG.md Records behavior changes and the full-build requirement.

Review effort: Balanced

Comment thread code_review_graph/graph.py Outdated
Comment on lines +1679 to +1683
owners = built(test) & {
owner(c) for c in candidates if isinstance(c, str)
}
if owners and owner(symbol_qn) not in owners:
continue

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 923962a: a class built on the call's own line now settles only that call, so app(SyncBooks::class)->handle() keeps its guess; a class built on a line with no such call, as in $job = new X() and a later $job->withBatchId($id)->handle() (seen on monica), still settles the others. Both shapes have a test, and guess counts on koel and monica are unchanged.

A test that ran (new UpdateBooks)->handle() and then
app(SyncBooks::class)->handle() lost the guess for SyncBooks::handle:
any class the test built hid the guess for every ambiguous call in it.

A class built on a call's own line now settles only that call. A class
built on a line with no such call, as in $job = new X() and a later
$job->withBatchId($id)->handle() (seen on monica), still settles the
test's other calls. Guess counts on koel and monica are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants