Repository navigation
feat(build): wire Jedi enrichment into builds; fix impact-accuracy benchmark - #823
3 commits merged into
Conversation
…nchmark - full_build and incremental_update now run Jedi call-resolution enrichment after the other resolvers. Best-effort: skipped without the [enrichment] extra or with CRG_DISABLE_ENRICHMENT=1; stats surface as result['jedi_enrichment']. - jedi_resolver: rebase resolved target paths onto the repo root as given to full_build, so enriched CALLS edges match stored node qualified names (fixes /var vs /private/var mismatches). - impact_accuracy: normalize changed-file paths to the stored absolute form so seed/neighbor/flow lookups hit; count File-node-sourced (module-level and IMPORTS_FROM) edges as neighbor files. - tests: full_build/incremental_update wiring regression tests and the co-change mock guard.
…chment - Remove 'harness needs fixing' note (co-change mode now works) - Clarify why published eval CSVs show predicted_files=0 (non-code seeds) - Update limitations section to reflect working co-change mode
|
The wiring is close: the three new tests fail on main and pass here, the suite is green, and the /var vs /private/var rebase in jedi_resolver.py is correct. Two things need fixing. incremental.py line 1360 runs the whole-repo pass on any Python change. enrich_jedi_calls re-reads and re-parses every .py file in the graph and retries goto on every unresolved call, so cost scales with repo size, not change size. A one-file update on this repo goes from about 0s to 1.1s with jedi installed; a few thousand Python files means tens of seconds per watch-mode save. Scope it to the changed files or keep it full-build only. README.md line 263 claims the published predicted_files = 0 rows came from documentation seeds. The 2026-08-02 CSVs show code seeds at 0: code_review_graph/cli.py twice, gin context.go, tree.go, render/data.go. Those zeros were the path bug. Also mark the recall 1.000 table at line 261 as pre-fix, since the new ground truth cannot reproduce it. The benchmark fix itself looks right. |
|
Changes required: enabling Jedi enrichment makes a one-file Python update read and reparse every indexed Python file. With the |
|
This no longer merges into git fetch origin && git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git pushWhen resolving:
A large integration branch landed on PRs now target |
|
This no longer merges into
Worth knowing before you resolve:
git fetch origin
git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git pushI have not reviewed the change itself yet. That comes once it merges and the checks run against the merged state, since staging has moved a long way and the result is what matters. |
|
Thank you for this PR. I tested it on a small repo and on a merge with staging. Before I can merge it, please fix these:
Please update your branch with the latest staging. There are conflicts in After you push the fix, I will test it again. |
…ed paths Addresses the review on tirth8205#823: - Incremental updates no longer make Jedi read every Python file. enrich_jedi_calls takes an optional files scope; incremental_update passes only the Python files it re-parsed. Full builds still scan all. - README: the co-change zeros were caused by the relative/absolute path bug, not doc seeds (code seeds such as cli.py and gin context.go were zero too). Note that the recall table predates the fix. - Co-change rows record seed_file relative to the repo root instead of the local clone path. - Merged latest staging. The Jedi step runs before _refresh_target_resolution, which stays last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GDCA6EwHMWTPTSxZGBYHfa
|
Thanks for the review. All three are fixed and the branch is updated with staging:
The Jedi step runs before |
There was a problem hiding this comment.
🟡 Changes recommended
Jedi enrichment still skips unresolved calls and leaves dependent callers’ edges stale, while non-canonical benchmark roots cause lookup misses and empty co-change predictions.
3 open issues
What changed in this PR
This PR integrates optional Jedi call resolution into graph builds and corrects path handling in the impact-accuracy benchmark.
Changes:
- Adds best-effort Jedi enrichment, an environment opt-out, build statistics, and incremental scan scoping.
- Updates benchmark path normalization and neighbor detection while preserving relative seed paths in results.
- Adds regression tests and documents that published benchmark figures need a fresh capture.
| File | Description |
|---|---|
tests/test_incremental.py |
Tests Jedi wiring, opt-out behavior, statistics, and scan scope. |
tests/test_eval.py |
Checks non-zero co-change predictions and relative seed reporting. |
README.md |
Explains the benchmark path bug and qualifies historical results. |
code_review_graph/jedi_resolver.py |
Adds file-scoped scans and rebases resolved target paths. |
code_review_graph/incremental.py |
Invokes Jedi enrichment during full and incremental builds. |
code_review_graph/eval/benchmarks/impact_accuracy.py |
Normalizes lookup paths and includes File-node sources in neighbors. |
Review effort: Balanced
| # Join against the root exactly as handed to full_build: the runner | ||
| # resolves the clone path once and builds the graph against it, so any | ||
| # resolution done here must match the stored node paths. Forcing a | ||
| # resolve() here would break graphs built from a non-canonical root | ||
| # (e.g. macOS /var vs /private/var symlinks). | ||
| root = Path(repo_path) |
| temporal_stats = _run_temporal_resolver(store) | ||
| hcl_stats = _run_hcl_resolver(store) | ||
| scoped_stats = _run_scoped_resolver(store) | ||
| jedi_stats = _run_jedi_enrichment(store, repo_root) |
| reparsed_py = [p for p in reparsed_paths if p.endswith(".py")] | ||
| jedi_stats = ( | ||
| _run_jedi_enrichment(store, repo_root, files=reparsed_py) | ||
| if reparsed_py else None |
Jedi adds about a second per Python file, so having the enrichment extra installed (for example through [all]) no longer turns it on. build and update take --enrich, and CRG_ENABLE_ENRICHMENT=1 asks for it where there is no flag (hooks, the watcher, MCP builds). CRG_DISABLE_ENRICHMENT=1 still keeps it off. Without a request Jedi is not imported and the graph is the one staging builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d01c1fd
… (review of tirth8205#823) The parser now keeps s.authenticate() as a CALLS edge to the bare name, and the Jedi step skipped any call with an edge on its line, so it resolved nothing. A bare edge now gets the Jedi target in place, never a target the graph has no node for. Dependents that update re-parses for their edges only lose their Jedi targets, so they are scanned again too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…irth8205#823) full_build stores paths under the resolved root, so a symlinked or relative repo path matched no node and co-change predictions were empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…w of tirth8205#823) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tirth8205#1139 loads the tool modules on first use. In test_cli.py an update test patches GraphStore and get_db_path while they load for the first time, so the mocks stayed bound in them and the real search test from tirth8205#823 that ran next found nothing. The test conftest now loads them up front. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Summary
This PR wires the
[enrichment]extra (Jedi call-resolution) into the build pipeline and fixes the co-change benchmark mode that was always returningpredicted_files = 0.Changes
Core wiring (
code_review_graph/incremental.py)_run_jedi_enrichment(store, repo_root)— best-effort wrapper matching the resolver patternfull_build(always) andincremental_update(when Python files changed)result["jedi_enrichment"](e.g.{"resolved": 5, "files": 1})CRG_DISABLE_ENRICHMENT=1(keeps watch mode cheap on large repos)Bug fix in
code_review_graph/jedi_resolver.pyfull_build(not force-resolved)/varvs/private/varsymlink mismatches where enriched edges never matched stored node qualified namesBenchmark fix (
code_review_graph/eval/benchmarks/impact_accuracy.py)0on every commit)_graph_neighbor_filesnow counts File-node-sourced edges (IMPORTS_FROM, module-level calls) as neighbor filesTests (
tests/test_incremental.py,tests/test_eval.py)test_full_build_wires_jedi_enrichment— verifies wiring + stats surfacetest_full_build_jedi_enrichment_skips_when_disabled— opt-out workstest_incremental_update_wires_jedi_enrichment— runs only on Python changespredicted_files > 0andtrue_positives >= 1for co-changeVerification
Jedi enrichment: resolved 5 calls in 1 filesNotes
[enrichment]extra (jedi>=0.19.2) — optional, skipped gracefully