Skip to content

feat(build): wire Jedi enrichment into builds; fix impact-accuracy benchmark - #823

Merged
3 commits merged into
tirth8205:stagingfrom
B1ZZ211:feat/jedi-enrichment-wiring
Nov 9, 2026
Merged

3 commits merged into
tirth8205:stagingfrom
B1ZZ211:feat/jedi-enrichment-wiring

Conversation

@B1ZZ211

@B1ZZ211 B1ZZ211 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR wires the [enrichment] extra (Jedi call-resolution) into the build pipeline and fixes the co-change benchmark mode that was always returning predicted_files = 0.

Changes

Core wiring (code_review_graph/incremental.py)

  • Added _run_jedi_enrichment(store, repo_root) — best-effort wrapper matching the resolver pattern
  • Called in full_build (always) and incremental_update (when Python files changed)
  • Stats surfaced as result["jedi_enrichment"] (e.g. {"resolved": 5, "files": 1})
  • Opt-out via CRG_DISABLE_ENRICHMENT=1 (keeps watch mode cheap on large repos)

Bug fix in code_review_graph/jedi_resolver.py

  • Target paths now rebase onto the repo root as given to full_build (not force-resolved)
  • Fixes /var vs /private/var symlink mismatches where enriched edges never matched stored node qualified names

Benchmark fix (code_review_graph/eval/benchmarks/impact_accuracy.py)

  • Normalizes changed-file paths to the stored absolute form before lookups
  • Co-change mode now emits honest non-zero predictions (was 0 on every commit)
  • Graph-derived ground truth now includes real one-hop neighbors (recall drops from pinned 1.0 to measured 0.667 where neighbors exist)
  • _graph_neighbor_files now counts File-node-sourced edges (IMPORTS_FROM, module-level calls) as neighbor files

Tests (tests/test_incremental.py, tests/test_eval.py)

  • test_full_build_wires_jedi_enrichment — verifies wiring + stats surface
  • test_full_build_jedi_enrichment_skips_when_disabled — opt-out works
  • test_incremental_update_wires_jedi_enrichment — runs only on Python changes
  • Mock regression test asserts predicted_files > 0 and true_positives >= 1 for co-change

Verification

  • Full test suite: 2400 passed (1 pre-existing Windows test failure, unrelated)
  • Build output: Jedi enrichment: resolved 5 calls in 1 files
  • Co-change mode on real repos now reports honest numbers (previously always 0)
  • All changes follow existing resolver patterns and conventions

Notes

  • Requires [enrichment] extra (jedi>=0.19.2) — optional, skipped gracefully
  • The published eval numbers in README are now stale; a fresh 6-repo capture would be needed to re-quote them

MrB1ZZ added 2 commits August 4, 2026 05:24
…nchmark

- full_build and incremental_update now run Jedi call-resolution
  enrichment after the other resolvers. Best-effort: skipped without the
  [enrichment] extra or with CRG_DISABLE_ENRICHMENT=1; stats surface as
  result['jedi_enrichment'].
- jedi_resolver: rebase resolved target paths onto the repo root as given
  to full_build, so enriched CALLS edges match stored node qualified
  names (fixes /var vs /private/var mismatches).
- impact_accuracy: normalize changed-file paths to the stored absolute
  form so seed/neighbor/flow lookups hit; count File-node-sourced
  (module-level and IMPORTS_FROM) edges as neighbor files.
- tests: full_build/incremental_update wiring regression tests and the
  co-change mock guard.
…chment

- Remove 'harness needs fixing' note (co-change mode now works)
- Clarify why published eval CSVs show predicted_files=0 (non-code seeds)
- Update limitations section to reflect working co-change mode
@tirth8205

Copy link
Copy Markdown
Owner

The wiring is close: the three new tests fail on main and pass here, the suite is green, and the /var vs /private/var rebase in jedi_resolver.py is correct. Two things need fixing.

incremental.py line 1360 runs the whole-repo pass on any Python change. enrich_jedi_calls re-reads and re-parses every .py file in the graph and retries goto on every unresolved call, so cost scales with repo size, not change size. A one-file update on this repo goes from about 0s to 1.1s with jedi installed; a few thousand Python files means tens of seconds per watch-mode save. Scope it to the changed files or keep it full-build only.

README.md line 263 claims the published predicted_files = 0 rows came from documentation seeds. The 2026-08-02 CSVs show code seeds at 0: code_review_graph/cli.py twice, gin context.go, tree.go, render/data.go. Those zeros were the path bug. Also mark the recall 1.000 table at line 261 as pre-fix, since the new ground truth cannot reproduce it.

The benchmark fix itself looks right.

@tirth8205

Copy link
Copy Markdown
Owner

Changes required: enabling Jedi enrichment makes a one-file Python update read and reparse every indexed Python file. With the [enrichment] extra installed, build a multi-file Python graph, edit one file and run code-review-graph update; enrich_jedi_calls still enumerates store.get_all_files() and reads and parses every Python file before selecting pending method calls. Limit incremental enrichment to changed files before claiming a coding-session efficiency improvement.

@tirth8205
tirth8205 changed the base branch from main to staging September 15, 2026 13:12
@tirth8205

tirth8205 commented Sep 15, 2026 •

Copy link
Copy Markdown
Owner

This no longer merges into staging. Conflicts in README.md, tests/test_incremental.py.

git fetch origin && git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git push

When resolving:

  • Rebase feat/jedi-enrichment-wiring onto current origin/staging and resolve the two conflicts before any gate or edge-case assessment can be made.

A large integration branch landed on staging today, which is why this drifted.

PRs now target staging, not main. Yours was retargeted already, so nothing to do there.

@tirth8205 tirth8205 added the needs-rebase Branch no longer merges into staging label Sep 15, 2026
@tirth8205

Copy link
Copy Markdown
Owner

This no longer merges into staging. 3 files conflict and the branch is 313 commits behind:

  • README.md
  • code_review_graph/incremental.py
  • tests/test_incremental.py

Worth knowing before you resolve:

  • code_review_graph/incremental.py: staging added a build-state checkpoint at the tail of full_build and moved the Git timeout into constants.py.
git fetch origin
git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git push

I have not reviewed the change itself yet. That comes once it merges and the checks run against the merged state, since staging has moved a long way and the result is what matters.

@tirth8205

Copy link
Copy Markdown
Owner

Thank you for this PR. I tested it on a small repo and on a merge with staging.

Before I can merge it, please fix these:

  1. Each incremental update reads every Python file again. See code_review_graph/incremental.py:1360. enrich_jedi_calls loops over store.get_all_files(). Example: a repo has 122 Python files. You edit one file and run update. Jedi then reads all 122 files. Please scan only the changed files.
  2. The new text at README.md:263 and README.md:283 is wrong. It says doc seeds caused the zeros. But code seeds were zero too, like code_review_graph/cli.py and gin context.go. See evaluate/results/*_impact_accuracy_2026-08-02.csv. The real cause was the path bug you fixed. Please also say the recall table is from before this fix.
  3. Co-change rows now save a full local path in seed_file. See impact_accuracy.py:186. Example: /tmp/.../mock_repo/helper.py instead of helper.py. Please keep the path relative to the repo root.

Please update your branch with the latest staging. There are conflicts in README.md, code_review_graph/incremental.py and tests/test_incremental.py. On staging, _refresh_target_resolution(store) must run last. Please call the Jedi step before it.

After you push the fix, I will test it again.

…ed paths

Addresses the review on tirth8205#823:

- Incremental updates no longer make Jedi read every Python file.
  enrich_jedi_calls takes an optional files scope; incremental_update
  passes only the Python files it re-parsed. Full builds still scan all.
- README: the co-change zeros were caused by the relative/absolute path
  bug, not doc seeds (code seeds such as cli.py and gin context.go were
  zero too). Note that the recall table predates the fix.
- Co-change rows record seed_file relative to the repo root instead of
  the local clone path.
- Merged latest staging. The Jedi step runs before
  _refresh_target_resolution, which stays last.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GDCA6EwHMWTPTSxZGBYHfa
@B1ZZ211

B1ZZ211 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All three are fixed and the branch is updated with staging:

  1. Incremental updates now pass only the re-parsed Python files to Jedi; full builds still scan all.
  2. README now attributes the zeros to the path bug (code seeds were zero too) and notes the recall table predates the fix.
  3. seed_file is now repo-relative.

The Jedi step runs before _refresh_target_resolution, which stays last.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Jedi enrichment still skips unresolved calls and leaves dependent callers’ edges stale, while non-canonical benchmark roots cause lookup misses and empty co-change predictions.

3 open issues
What changed in this PR

This PR integrates optional Jedi call resolution into graph builds and corrects path handling in the impact-accuracy benchmark.

Changes:

  • Adds best-effort Jedi enrichment, an environment opt-out, build statistics, and incremental scan scoping.
  • Updates benchmark path normalization and neighbor detection while preserving relative seed paths in results.
  • Adds regression tests and documents that published benchmark figures need a fresh capture.
File Description
tests/​test_incremental.py Tests Jedi wiring, opt-out behavior, statistics, and scan scope.
tests/​test_eval.py Checks non-zero co-change predictions and relative seed reporting.
README.md Explains the benchmark path bug and qualifies historical results.
code_review_graph/​jedi_resolver.py Adds file-scoped scans and rebases resolved target paths.
code_review_graph/​incremental.py Invokes Jedi enrichment during full and incremental builds.
code_review_graph/​eval/​benchmarks/​impact_accuracy.py Normalizes lookup paths and includes File-node sources in neighbors.

Review effort: Balanced

Comment on lines +139 to +144
# Join against the root exactly as handed to full_build: the runner
# resolves the clone path once and builds the graph against it, so any
# resolution done here must match the stored node paths. Forcing a
# resolve() here would break graphs built from a non-canonical root
# (e.g. macOS /var vs /private/var symlinks).
root = Path(repo_path)
temporal_stats = _run_temporal_resolver(store)
hcl_stats = _run_hcl_resolver(store)
scoped_stats = _run_scoped_resolver(store)
jedi_stats = _run_jedi_enrichment(store, repo_root)
Comment on lines +2157 to +2160
reparsed_py = [p for p in reparsed_paths if p.endswith(".py")]
jedi_stats = (
_run_jedi_enrichment(store, repo_root, files=reparsed_py)
if reparsed_py else None
tirth8205 added a commit that referenced this pull request Nov 9, 2026
Jedi adds about a second per Python file, so having the enrichment extra
installed (for example through [all]) no longer turns it on. build and
update take --enrich, and CRG_ENABLE_ENRICHMENT=1 asks for it where there
is no flag (hooks, the watcher, MCP builds). CRG_DISABLE_ENRICHMENT=1
still keeps it off. Without a request Jedi is not imported and the graph
is the one staging builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tirth8205 added a commit that referenced this pull request Nov 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tirth8205 tirth8205 closed this pull request by merging all changes into tirth8205:staging in d01c1fd Nov 9, 2026
HengYangDS pushed a commit to HengYangDS/code-review-graph that referenced this pull request Nov 9, 2026
… (review of tirth8205#823)

The parser now keeps s.authenticate() as a CALLS edge to the bare name,
and the Jedi step skipped any call with an edge on its line, so it
resolved nothing. A bare edge now gets the Jedi target in place, never a
target the graph has no node for. Dependents that update re-parses for
their edges only lose their Jedi targets, so they are scanned again too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengYangDS pushed a commit to HengYangDS/code-review-graph that referenced this pull request Nov 9, 2026
…irth8205#823)

full_build stores paths under the resolved root, so a symlinked or
relative repo path matched no node and co-change predictions were empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengYangDS pushed a commit to HengYangDS/code-review-graph that referenced this pull request Nov 9, 2026
…w of tirth8205#823)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HengYangDS pushed a commit to HengYangDS/code-review-graph that referenced this pull request Nov 9, 2026
tirth8205#1139 loads the tool modules on first use. In test_cli.py an update test
patches GraphStore and get_db_path while they load for the first time, so
the mocks stayed bound in them and the real search test from tirth8205#823 that ran
next found nothing. The test conftest now loads them up front.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase Branch no longer merges into staging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants