Repository navigation
feat(affected): rank results by distance, then link strength, then path - #5069
rohit-jsfreaky wants to merge 2 commits into
Conversation
`affected` returned hits in breadth-first discovery order, which within a depth is just graph-iteration order. On a hub file the first lines were whichever dependents happened to be stored first (django/conf/__init__.py has 938 dependent files at depth 2), so the top of the list said little. Hits are now sorted by: 1. distance from the seed (depth), as before; 2. how strongly the hit's file links one step closer to the seed: edges, over the traversed relations, from that file into the seed's file (depth 1) or into the files first reached at depth - 1; more first; 3. file path, then line, then node id, so ties break the same way every run whatever order the graph was built in. The result set, depths and relations are unchanged; only the order is. Measured on 2,070 queries replayed from six repos (files changed in the same commit as truth), recall@10 goes from 0.179 to 0.221 and hit@10 from 0.340 to 0.395 (122 vs 8 discordant queries). On 107 mutated functions in click, flask and httpx (failing test files as truth) the effect is small: recall@10 0.164 to 0.170, hit@10 0.374 to 0.421, not significant. `affected` has no JSON output and no MCP tool on v8, so only the CLI text changes; any later consumer of affected_nodes gets the same order. Co-Authored-By: Claude <noreply@anthropic.com>
|
Thanks for the pull request, @rohit-jsfreaky. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic. |
github-actions
Bot
added
the
contributor
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 2 changed function(s) (2 sampled, not proven).
Formal verification. PR-changed functions: 2/2 verified (0 proven, 2 may-equivalent, 0 distinguished) · 0 not verified.
Graphify review — findings
Ranks affected output so the most likely impacted hits come first: nearest depth, then files with the most edges toward the seed (or the previous depth's files), then path, line and id as a stable tiebreak. The result set and depths are unchanged, and the order no longer depends on graph build order or hash seed. Link counting treats the seed's file as the depth-1 target, so a seed with no source_file can match unrelated unfiled nodes and inflate those counts. The CLI help line now notes the new ordering.
No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 516 functions depend on the 68 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
main()— 123 callers, 3 callees - worse:
affected_nodes()— 39 callers, 3 callees
Verification — 516 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 490 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
59 of 527 test file(s) selected (11%) via static blast radius.
tests/test_4874_jvm_resolution.py— impacttests/test_affected_cli.py— impacttests/test_affected_defines.py— impacttests/test_affected_member_seed.py— impacttests/test_affected_overrides.py— impacttests/test_affected_path_file_node.py— impacttests/test_affected_ranking.py— impact, changed-testtests/test_affected_seed_reference_forms.py— impacttests/test_agents_platform.py— impacttests/test_benchmark.py— impacttests/test_benchmark_raw_graph.py— impacttests/test_benchmark_real_query.py— impacttests/test_codebuddy.py— impacttests/test_confidence_score_output.py— impacttests/test_corrupt_graph_json.py— impacttests/test_dedup_shrink_refuses_force_write.py— impacttests/test_devin.py— impacttests/test_edge_also_seen.py— impacttests/test_explain_cli.py— impacttests/test_explain_same_file_tie.py— impacttests/test_extract.py— impacttests/test_extract_cli.py— impacttests/test_extract_cost_ledger.py— impacttests/test_global_add_tag_inference.py— impacttests/test_god_nodes_cli.py— impacttests/test_hollow_chunks_arm_shrink_guard.py— impacttests/test_incomplete_build_guard.py— impacttests/test_indirect_call_nested_closure_shadow.py— impacttests/test_indirect_dispatch.py— impacttests/test_indirect_dispatch_assign_return.py— impacttests/test_indirect_dispatch_getattr.py— impacttests/test_install.py— impacttests/test_install_references.py— impacttests/test_install_version_warning.py— impacttests/test_js_dynamic_import_affected.py— impacttests/test_js_dynamic_imports.py— impacttests/test_lookup_exit_codes.py— impacttests/test_mcp_graph_view_cache.py— impacttests/test_merge_chunks_validation.py— impacttests/test_multigraph_diagnostics.py— impacttests/test_no_dedup_flag.py— impacttests/test_opencode_plugin_refresh.py— impacttests/test_path_cli.py— impacttests/test_path_confidence.py— impacttests/test_path_type_member.py— impacttests/test_query_cli.py— impacttests/test_query_induced_edges.py— impacttests/test_query_mcp_direction.py— impacttests/test_query_names_its_graph.py— impacttests/test_query_seed_order.py— impact- … and 9 more
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Docs that may be stale (advisory)
BENCHMARKS.md§ Harness (lines 37-57): references changed symbolskeyCHANGELOG.md§ 0.9.93 (2026-10-20) (lines 243-269): references changed symbolskeyCHANGELOG.md§ 0.9.70 (2026-09-27) (lines 648-657): references changed symbolskeyCHANGELOG.md§ 0.9.31 (2026-07-30) (lines 1121-1131): references changed symbolskeyCHANGELOG.md§ 0.8.49 (2026-06-24) (lines 1616-1624): references changed symbolskeyCHANGELOG.md§ 0.8.34 (2026-06-07) (lines 1776-1793): references changed symbolskeyCHANGELOG.md§ 0.8.6 (2026-05-16) (lines 2016-2024): references changed symbolskeyREADME.md§ (preamble) (lines 1-79): references changed symbolskeyREADME.md§ Project configuration (lines 397-430): references changed symbolskeydocs/superpowers/plans/2026-05-04-incremental-updates-dedup.md§ Task 2: Create graphify/dedup.py — entropy gate + MinHash/LSH + Jaro-Winkler (lines 71-468): references changed symbolskey
…and 10 more.
Formal verification
No difference found (not proven): No behavior difference found in \_run\_cli (not a proof).
The verifier ran both versions of \_run\_cli on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
No difference found (not proven): No behavior difference found in affected\_nodes (not a proof).
The verifier ran both versions of affected\_nodes on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
· 2 more finding(s) on lines outside this diff (see the check run).
…file When the seed has no `source_file`, its "file" was the empty string, so every edge into any other unfiled node counted as a link toward the seed and could push a hit up the list. The empty file is now never a target. Co-Authored-By: Claude <noreply@anthropic.com>
|
Thanks, the seed-without-a-file point was real. Fixed in da6be38: when the seed has no |
Closes #5067
What does this PR do?
affectedreturns the same hits as before, sorted so the most likely impacted come first:Hits of one file stay together within a depth. The set and depth never change; only the order does.
zzz.pycalls intocore.pyfour times;aaa.pyonly imports it.affectedhas no JSON output or MCP tool on v8, so the CLI text is the only output that changes.Diff:
graphify/affected.py+78/-2 (rank_hits, called at the end ofaffected_nodes), one help line, 6 tests.Numbers
Git-history replay on six repos (graph at a cut commit about 6 months before HEAD, 2,070 later queries; seed = a changed code file, truth = the other files in that commit). The result set is identical for all 2,070 queries. recall@10 / hit@10:
Paired hit@10: 122 queries only this PR gets right, 8 only v8.
Failing tests (107 mutated functions in click, flask and httpx, failing test files = truth): about neutral. recall@10 0.164 to 0.170, hit@10 0.374 to 0.421 (8 vs 3, p = 0.23); httpx recall@20 drops from 0.209 to 0.174.
Limitations
affectedtext and rely on the old order will see a different order.Related
#1478 adds an opt-in weighted traversal with hub penalties, which can change which nodes come back. This PR has no options and keeps the set. #5068 (co-change in
affected) also editsaffected.py; the two merge cleanly, and with both, the JSON there follows this order.Type of change
Verification & Invariants
Invariant: the hit set and each hit's depth are unchanged; only order changes, and it does not depend on graph insertion order or
PYTHONHASHSEED. Tests check the set and depth against v8's, the exact order, tie breaking by path, insertion-order independence, CLI text order and hash-seed stability. No persisted state changes.How was this tested?
Graphify-specific checklist
uv run python -m tools.skillgen --bless) when changing their source fragments.