Repository navigation
feat: add opt-in line-oriented graph.json output - #5105
qq2514568380 wants to merge 1 commit into
Conversation
A committed graph.json is one compact line (Graphify-Labs#3413), which breaks line-oriented tools and makes every refresh a full-file diff. Add GRAPHIFY_GRAPH_JSON_LINES to write the same graph with one node or link per line, sorted by id and by (source, target, relation), with a trailing newline. All project graph writers share the new writer: update, extract --no-cluster, merge-graphs, the merge driver and stale-source pruning. The default stays the single-line form. The layout flag is part of the build state, so enabling or disabling it on an unchanged tree rewrites graph.json once and is idempotent afterwards. Fixes Graphify-Labs#4942
|
Thanks for the pull request, @qq2514568380. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic. |
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 5 changed function(s) (1 sampled, not proven, 4 not verified).
Formal verification. PR-changed functions: 1/5 verified (0 proven, 1 may-equivalent, 0 distinguished) · 4 not verified (2 vacuous, 2 unsupported).
Not verified on this run: \_prune\_graph\_json\_sources (unsupported), dispatch\_command (vacuous: never exercised), \_current\_build\_state (vacuous: never exercised), \_rebuild\_code (unsupported).
Graphify review — findings
Adds an opt-in line-oriented graph.json layout behind GRAPHIFY_GRAPH_JSON_LINES, with one node, link or hyperedge per line, sorted keys and elements, and a trailing newline. Every graph writer (the CLI prune, merge and extract paths and to_json) now goes through write_graph_json_atomic, which falls back to the existing single compact line when the flag is off. Records graph_json_lines in the watch build state, and _rebuild_code falls through to a full extraction when only that setting changed, so the file is rewritten in the requested layout instead of skipping on "no tracked code files". A build state saved before this field existed compares unequal to the current setting, so the first rebuild after upgrade can trigger a full re-extraction that wasn't needed.
No blocking issues surfaced. 4 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 4764 functions depend on the 334 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
_rebuild_code()— 270 callers, 76 callees - worse:
to_json()— 106 callers, 13 callees - worse:
dispatch_command()— 8 callers, 149 callees - new:
write_graph_json_atomic()— 5 callers, 4 callees
Verification — 4764 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 3516 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
224 of 533 test file(s) selected (42%) via static blast radius.
tests/test_4874_jvm_resolution.py— impacttests/test_4978_reconcile_cost.py— impacttests/test_affected_cli.py— impacttests/test_affected_overrides.py— impacttests/test_affected_path_file_node.py— impacttests/test_affected_seed_reference_forms.py— impacttests/test_agents_platform.py— impacttests/test_alias_folding.py— impacttests/test_analyze.py— impacttests/test_atomic_canvas_export.py— impacttests/test_atomic_version_stamp.py— impacttests/test_atomic_writes.py— impacttests/test_benchmark.py— impacttests/test_benchmark_raw_graph.py— impacttests/test_benchmark_real_query.py— impacttests/test_binary_media_words.py— impacttests/test_build.py— impacttests/test_build_located_semantic_identity.py— impacttests/test_build_merge_dedup_scope.py— impacttests/test_build_merge_ghost_fold_4955.py— impacttests/test_build_merge_hyperedges_and_prune.py— impacttests/test_build_merge_shrink_guard.py— impacttests/test_c_include_resolution.py— impacttests/test_cache.py— impacttests/test_cache_long_paths.py— impacttests/test_cache_stale_import_target.py— impacttests/test_cache_windows_case_4925.py— impacttests/test_callflow_html.py— impacttests/test_carried_hyperedge_remap.py— impacttests/test_charmap_encoding.py— impacttests/test_chunking.py— impacttests/test_cli_export.py— impacttests/test_cloud_cta.py— impacttests/test_cluster.py— impacttests/test_code_only_scope_4923.py— impacttests/test_codebuddy.py— impacttests/test_community_labels_skill.py— impacttests/test_compact_graph_json.py— impacttests/test_confidence.py— impacttests/test_confidence_score_output.py— impacttests/test_corrupt_graph_json.py— impacttests/test_cpp_objc_cross_file_calls.py— impacttests/test_cross_extension_reexport_self_cycle.py— impacttests/test_csharp_method_owner_labels.py— impacttests/test_csharp_object_creation.py— impacttests/test_csharp_partial_classes.py— impacttests/test_dedup.py— impacttests/test_dedup_remaps_hyperedges.py— impacttests/test_dedup_shrink_refuses_force_write.py— impacttests/test_definition_file_portability.py— impact- … and 174 more
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Could not verify: Could not verify \_prune\_graph\_json\_sources.
The verifier did not have enough to check \_prune\_graph\_json\_sources, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: the input domain has 83 values but only 11 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 40 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in to\_json (not a proof).
The verifier ran both versions of to\_json on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_current\_build\_state.
The verifier did not have enough to check \_current\_build\_state, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_rebuild\_code.
The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: nondeterministic output (both versions disagreed with themselves on the witness) — not a behaviour change; a function whose output is not a function of its inputs (random / time / uuid / os.urandom, or state carried across calls) cannot be soundly distinguished by execution, so this tier abstains for the whole function
· 1 grounded finding(s) anchored inline below; 3 more finding(s) on lines outside this diff (see the check run).
| return json.dumps(obj, ensure_ascii=False, separators=(",", ":")) + "\n" | ||
|
|
||
|
|
||
| def write_graph_json_atomic(path: "str | Path", obj: dict) -> None: |
There was a problem hiding this comment.
write_graph_json_atomic()
high coupling complexity (Ca·Ce = 20).
Grounded coupling-delta finding (deterministic), not an LLM guess.
df860db to
4b62160
Compare
Summary
Fixes #4942. Adds an opt-in line-oriented layout for
graph.json: one node or link per line, deterministic order, same JSON content.Why opt-in
#3413 deliberately made
graph.jsona single compact line so a committed graph does not report tens of thousands of changed lines. This change keeps that default and puts the line-oriented form behind:GRAPHIFY_GRAPH_JSON_LINES=1 graphify update .Setting
1,trueoryesenables it for every project-graph writer, including hook-driven updates. Unset (or0/false) keeps the existing single-line output byte-for-byte.What changed
graphify/paths.py: newgraph_json_text()andwrite_graph_json_atomic(). In line mode, top-level keys are sorted,nodesbyid,linksby(source, target, relation), andhyperedgesby their canonical record. Each element is a compact record on its own line; the file ends with a newline and remains valid JSON.graphify/export.py,graphify/cli.py,graphify/watch.py: all project-graph writers now share that writer —update,extract --no-cluster,merge-graphs, the merge driver, and stale-source pruning.graphify/watch.py: the layout flag is part of the recorded build state, so enabling or disabling it on an unchanged tree still rewritesgraph.jsononce. After that, an unchanged line-oriented graph is left untouched (verified by inode/mtime). Switching back to compact also rewrites once.Tests
tests/test_graph_json_lines.py:falsekeeps the default;Manual smoke on a two-function corpus:
graphify updateand--no-cluster: 1 line;GRAPHIFY_GRAPH_JSON_LINES=1: 16 (clustered) / 17 (--no-cluster) lines;The full suite on this Windows host has the existing Windows/hook/wiki/install failures; the touched writer paths (
test_compact_graph_json.py,test_atomic_writes.py) pass. The onetest_merge_graphs_cli.pyfailure reproduces on the unmodifiedv8baseline.AI disclosure
Implemented with Claude Code; the commit carries the
Co-Authored-Bymetadata requested inCONTRIBUTING.md.