fix(query): order the per-term guaranteed seeds by match strength (#4357) by devkancheti4-design · Pull Request #4358 · Graphify-Labs/graphify

devkancheti4-design · 2026-10-12T07:00:59Z

What does this PR do?

Closes #4357.

_pick_seeds guarantees one seed per query term (#1445), but it appended those seeds in sorted(best_seed_by_term),
which is alphabetical term order. _subgraph_to_text prints seeds first, so on a multi-term query a weak match on an
early-alphabet term was printed ahead of a specific term's exact match.

_score_query already ranks each term's winner with the key (-singleton score, -degree, label length, node id),
and the singleton score carries the term's IDF weight. This PR:

returns that order as _QueryScores.term_order (the term itself breaks remaining ties);
has _query_graph_text pass it to _pick_seeds.

What stays the same:

The same seeds are picked, and BFS reaches the same nodes. Only the seeds' order changes.
The order is deterministic.
Without term_order, as with explain, path and direct callers, _pick_seeds keeps the sorted order.
Terms missing from term_order follow it in sorted order.

Measured on AST-only graphs with graphify's own functions (graphify 0.9.86, v8 a5ba60d):

set	hit@3 v8 / PR	hit@5 v8 / PR
Django issues from SWE-bench Lite, title + 400 chars (114)	53 / 58	53 / 63
Django issues, title only (114)	46 / 48	54 / 55
full-stack-fastapi-template, 100 questions	29 / 33	36 / 37

Per question, hit@5 moved +10/-0, +4/-3 and +1/-0. The
script and data reproduce
these numbers.

Type of change

Verification & Invariants

The invariants:

the seed set is unchanged;
the traversal is unchanged;
the output is deterministic;
callers that pass no term_order keep the legacy order.

No persisted or generated state is touched.

Read the CONTRIBUTING.md guide.
Reproduced the issue and identified the invariant.
Made the smallest fix necessary.
Added a regression test (if bug fix) or isolated boundary test.
Kept the PR description synchronized with the final implementation.
Documented any limitations / unsupported cases explicitly.

How was this tested?

uv run pytest -q tests/test_query_seed_order.py      5 passed (4 of them fail on v8)
uv run pytest -q tests/test_serve.py tests/test_query*.py tests/test_mcp*.py tests/test_affected*.py    285 passed
uv run pytest -q tests                               7,321 passed, 15 skipped, 0 failed
uvx ruff check graphify/serve.py tests/test_query_seed_order.py      All checks passed

What the new tests cover:

_pick_seeds follows term_order;
_pick_seeds keeps the sorted order without it;
terms missing from term_order follow it in sorted order;
_score_query orders terms by match strength;
end to end: the query alpha bravo charlie apple zebra printed the common term's seed (apple) before the rare
exact match (charlie) on v8, and prints charlie first with this change.

Graphify-specific checklist

I updated generated skill artifacts (uv run python -m tools.skillgen --bless) when changing their source fragments. (n/a, no fragments changed)
I confirmed that AST/structural extraction remains deterministic (no ambient state dependencies like ENV variables).
I reviewed changes for security implications (no unsafe interpolation into shell/Python).
I confirmed no API keys or local-only graph data are included.
(If applicable) I disclosed AI authorship in my commit messages.

🤖 Generated with Claude Code

…aphify-Labs#4357) _pick_seeds appended the per-term guaranteed seeds (Graphify-Labs#1445) in sorted(best_seed_by_term), alphabetical term order, and the renderer prints seeds first, so a weak match on an early-alphabet term led a specific term's exact match. _score_query now returns term_order: the per-term winners by their existing key (-singleton score, which carries the term's IDF weight, then -degree, label length, node id), the term breaking ties. _query_graph_text passes it to _pick_seeds. The same seeds are picked and the order is deterministic; without term_order the sorted order is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-12T07:01:07Z

Thanks for the pull request, @devkancheti4-design. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic.

graphify-labs

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; formal verification proved 0 of 3 changed function(s) (2 sampled, not proven, 1 not verified).

Formal verification. PR-changed functions: 2/3 verified (0 proven, 2 may-equivalent, 0 distinguished) · 1 not verified (1 vacuous).

Not verified on this run: \_score\_query (vacuous: never exercised).

Graphify review — findings

Orders the per-term guaranteed seeds in _pick_seeds by the strength of each term's best match (IDF-weighted score, then degree, label length, term) instead of alphabetically. A rare term's exact match now prints ahead of a common term's substring hit in the query output. _score_query returns this as term_order; terms missing from it, or callers that don't pass it, fall back to the old sorted order, and the set of seeds picked is unchanged.

No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 610 functions depend on the 158 functions this change touches.

Health — this change adds coupling hotspots:

worse: _query_graph_text() — 32 callers, 10 callees
worse: _score_query() — 17 callers, 7 callees

Verification — 610 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

verification_scope: 410 function(s) in the blast radius were not formally verified this run

Test selection

14 of 377 test file(s) selected (4%) via static blast radius.

tests/test_benchmark.py — impact
tests/test_benchmark_raw_graph.py — impact
tests/test_explain_same_file_tie.py — impact
tests/test_file_label_disambiguation.py — impact
tests/test_mcp_graph_view_cache.py — impact
tests/test_path_cli.py — impact
tests/test_query_induced_edges.py — impact
tests/test_query_mcp_direction.py — impact
tests/test_query_names_its_graph.py — impact
tests/test_query_seed_order.py — impact, changed-test
tests/test_serve.py — impact
tests/test_serve_http.py — impact
tests/test_terraform.py — impact
tests/test_truncation_full_answer.py — impact

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

No difference found (not proven): No behavior difference found in \_pick\_seeds (not a proof).

The verifier ran both versions of \_pick\_seeds on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

No difference found (not proven): No behavior difference found in \_query\_graph\_text (not a proof).

The verifier ran both versions of \_query\_graph\_text on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_score\_query.

The verifier did not have enough to check \_score\_query, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 47 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)

· 2 more finding(s) on lines outside this diff (see the check run).

safishamsi · 2026-10-12T14:21:49Z

Shipped in v0.9.87, just released to PyPI as graphifyy 0.9.87. Landed via an authorship-preserving cherry-pick (so the commit keeps your name and counts on your contributor graph) rather than a branch merge, which is why GitHub marks this closed rather than merged. Thanks @devkancheti4-design!

devkancheti4-design · 2026-10-12T19:12:03Z

Thank you @safishamsi, both for shipping it in 0.9.87 and for keeping the authorship on the cherry-pick.

devkancheti4-design requested a review from safishamsi as a code owner October 12, 2026 07:01

graphify-labs Bot reviewed Oct 12, 2026

View reviewed changes

safishamsi closed this Oct 12, 2026

This was referenced Oct 12, 2026

[Bug]: query prints per-term guaranteed seeds in alphabetical order, burying the answer file on multi-term queries #4357

Closed

feat(mcp): improve retrieval quality with phrase bonus, community coherence, and evidence-first truncation (#3810) #3866

Open

This was referenced Oct 15, 2026

feat(query): --compact writes the same answer in ~1.7x fewer tokens (#4469) #4470

Open

feat(query): --learned starts from nodes that past useful answers to a similar question cited (#4477) #4478

Open

Navigation Menu

Repository navigation

fix(query): order the per-term guaranteed seeds by match strength (#4357) - #4358

fix(query): order the per-term guaranteed seeds by match strength (#4357)#4358
devkancheti4-design wants to merge 1 commit into
Graphify-Labs:v8from
devkancheti4-design:seed-order

devkancheti4-design commented Oct 12, 2026

Uh oh!

github-actions Bot commented Oct 12, 2026

Uh oh!

graphify-labs Bot left a comment •

edited

Loading

Uh oh!

safishamsi commented Oct 12, 2026

Uh oh!

devkancheti4-design commented Oct 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

devkancheti4-design commented Oct 12, 2026

What does this PR do?

Type of change

Verification & Invariants

How was this tested?

Graphify-specific checklist

Uh oh!

github-actions Bot commented Oct 12, 2026

Uh oh!

graphify-labs Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Graphify review — findings

Impact & health

Graphify review

Gate & verification

graphify gate

Test selection

Test selection

Formal verification

Uh oh!

safishamsi commented Oct 12, 2026

Uh oh!

devkancheti4-design commented Oct 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

graphify-labs Bot left a comment •

edited

Loading