Skip to content

feat: add opt-in line-oriented graph.json output - #5105

Open
qq2514568380 wants to merge 1 commit into
Graphify-Labs:v8from
qq2514568380:feat/graph-json-lines
Open

qq2514568380 wants to merge 1 commit into
Graphify-Labs:v8from
qq2514568380:feat/graph-json-lines

Conversation

@qq2514568380

@qq2514568380 qq2514568380 commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #4942. Adds an opt-in line-oriented layout for graph.json: one node or link per line, deterministic order, same JSON content.

Why opt-in

#3413 deliberately made graph.json a single compact line so a committed graph does not report tens of thousands of changed lines. This change keeps that default and puts the line-oriented form behind:

GRAPHIFY_GRAPH_JSON_LINES=1 graphify update .

Setting 1, true or yes enables it for every project-graph writer, including hook-driven updates. Unset (or 0/false) keeps the existing single-line output byte-for-byte.

What changed

  • graphify/paths.py: new graph_json_text() and write_graph_json_atomic(). In line mode, top-level keys are sorted, nodes by id, links by (source, target, relation), and hyperedges by their canonical record. Each element is a compact record on its own line; the file ends with a newline and remains valid JSON.
  • graphify/export.py, graphify/cli.py, graphify/watch.py: all project-graph writers now share that writer — update, extract --no-cluster, merge-graphs, the merge driver, and stale-source pruning.
  • graphify/watch.py: the layout flag is part of the recorded build state, so enabling or disabling it on an unchanged tree still rewrites graph.json once. After that, an unchanged line-oriented graph is left untouched (verified by inode/mtime). Switching back to compact also rewrites once.

Tests

tests/test_graph_json_lines.py:

  • line mode writes one record per line, parses back to the same elements, and sorts nodes/links deterministically;
  • the default stays one line;
  • false keeps the default;
  • enabling the flag rewrites an existing compact graph into the line-oriented layout.
uv run pytest tests/test_graph_json_lines.py tests/test_compact_graph_json.py -q   # 16 passed
uv run ruff check <changed files>                                                 # passed
uv run pyright graphify/paths.py tests/test_graph_json_lines.py                    # 0 errors

Manual smoke on a two-function corpus:

  • default graphify update and --no-cluster: 1 line;
  • with GRAPHIFY_GRAPH_JSON_LINES=1: 16 (clustered) / 17 (--no-cluster) lines;
  • a second line-mode update leaves the file byte-identical (same inode, mtime and size);
  • compact → line and line → compact both rewrite once.

The full suite on this Windows host has the existing Windows/hook/wiki/install failures; the touched writer paths (test_compact_graph_json.py, test_atomic_writes.py) pass. The one test_merge_graphs_cli.py failure reproduces on the unmodified v8 baseline.

AI disclosure

Implemented with Claude Code; the commit carries the Co-Authored-By metadata requested in CONTRIBUTING.md.


A committed graph.json is one compact line (Graphify-Labs#3413), which breaks
line-oriented tools and makes every refresh a full-file diff. Add
GRAPHIFY_GRAPH_JSON_LINES to write the same graph with one node or link
per line, sorted by id and by (source, target, relation), with a trailing
newline.

All project graph writers share the new writer: update, extract
--no-cluster, merge-graphs, the merge driver and stale-source pruning. The
default stays the single-line form. The layout flag is part of the build
state, so enabling or disabling it on an unchanged tree rewrites graph.json
once and is idempotent afterwards.

Fixes Graphify-Labs#4942
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Thanks for the pull request, @qq2514568380. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic.

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; formal verification proved 0 of 5 changed function(s) (1 sampled, not proven, 4 not verified).

Formal verification. PR-changed functions: 1/5 verified (0 proven, 1 may-equivalent, 0 distinguished) · 4 not verified (2 vacuous, 2 unsupported).

Not verified on this run: \_prune\_graph\_json\_sources (unsupported), dispatch\_command (vacuous: never exercised), \_current\_build\_state (vacuous: never exercised), \_rebuild\_code (unsupported).


Graphify review — findings

Adds an opt-in line-oriented graph.json layout behind GRAPHIFY_GRAPH_JSON_LINES, with one node, link or hyperedge per line, sorted keys and elements, and a trailing newline. Every graph writer (the CLI prune, merge and extract paths and to_json) now goes through write_graph_json_atomic, which falls back to the existing single compact line when the flag is off. Records graph_json_lines in the watch build state, and _rebuild_code falls through to a full extraction when only that setting changed, so the file is rewritten in the requested layout instead of skipping on "no tracked code files". A build state saved before this field existed compares unequal to the current setting, so the first rebuild after upgrade can trigger a full re-extraction that wasn't needed.

No blocking issues surfaced. 4 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 4764 functions depend on the 334 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: _rebuild_code() — 270 callers, 76 callees
  • worse: to_json() — 106 callers, 13 callees
  • worse: dispatch_command() — 8 callers, 149 callees
  • new: write_graph_json_atomic() — 5 callers, 4 callees

Verification — 4764 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 3516 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

224 of 533 test file(s) selected (42%) via static blast radius.

  • tests/test_4874_jvm_resolution.py — impact
  • tests/test_4978_reconcile_cost.py — impact
  • tests/test_affected_cli.py — impact
  • tests/test_affected_overrides.py — impact
  • tests/test_affected_path_file_node.py — impact
  • tests/test_affected_seed_reference_forms.py — impact
  • tests/test_agents_platform.py — impact
  • tests/test_alias_folding.py — impact
  • tests/test_analyze.py — impact
  • tests/test_atomic_canvas_export.py — impact
  • tests/test_atomic_version_stamp.py — impact
  • tests/test_atomic_writes.py — impact
  • tests/test_benchmark.py — impact
  • tests/test_benchmark_raw_graph.py — impact
  • tests/test_benchmark_real_query.py — impact
  • tests/test_binary_media_words.py — impact
  • tests/test_build.py — impact
  • tests/test_build_located_semantic_identity.py — impact
  • tests/test_build_merge_dedup_scope.py — impact
  • tests/test_build_merge_ghost_fold_4955.py — impact
  • tests/test_build_merge_hyperedges_and_prune.py — impact
  • tests/test_build_merge_shrink_guard.py — impact
  • tests/test_c_include_resolution.py — impact
  • tests/test_cache.py — impact
  • tests/test_cache_long_paths.py — impact
  • tests/test_cache_stale_import_target.py — impact
  • tests/test_cache_windows_case_4925.py — impact
  • tests/test_callflow_html.py — impact
  • tests/test_carried_hyperedge_remap.py — impact
  • tests/test_charmap_encoding.py — impact
  • tests/test_chunking.py — impact
  • tests/test_cli_export.py — impact
  • tests/test_cloud_cta.py — impact
  • tests/test_cluster.py — impact
  • tests/test_code_only_scope_4923.py — impact
  • tests/test_codebuddy.py — impact
  • tests/test_community_labels_skill.py — impact
  • tests/test_compact_graph_json.py — impact
  • tests/test_confidence.py — impact
  • tests/test_confidence_score_output.py — impact
  • tests/test_corrupt_graph_json.py — impact
  • tests/test_cpp_objc_cross_file_calls.py — impact
  • tests/test_cross_extension_reexport_self_cycle.py — impact
  • tests/test_csharp_method_owner_labels.py — impact
  • tests/test_csharp_object_creation.py — impact
  • tests/test_csharp_partial_classes.py — impact
  • tests/test_dedup.py — impact
  • tests/test_dedup_remaps_hyperedges.py — impact
  • tests/test_dedup_shrink_refuses_force_write.py — impact
  • tests/test_definition_file_portability.py — impact
  • … and 174 more

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify \_prune\_graph\_json\_sources.

The verifier did not have enough to check \_prune\_graph\_json\_sources, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: the input domain has 83 values but only 11 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 40 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

No difference found (not proven): No behavior difference found in to\_json (not a proof).

The verifier ran both versions of to\_json on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_current\_build\_state.

The verifier did not have enough to check \_current\_build\_state, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_rebuild\_code.

The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: no capturable inputs from the test suite; property tier: nondeterministic output (both versions disagreed with themselves on the witness) — not a behaviour change; a function whose output is not a function of its inputs (random / time / uuid / os.urandom, or state carried across calls) cannot be soundly distinguished by execution, so this tier abstains for the whole function

· 1 grounded finding(s) anchored inline below; 3 more finding(s) on lines outside this diff (see the check run).

Comment thread graphify/paths.py
return json.dumps(obj, ensure_ascii=False, separators=(",", ":")) + "\n"


def write_graph_json_atomic(path: "str | Path", obj: dict) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — write_graph_json_atomic()

high coupling complexity (Ca·Ce = 20).

Grounded coupling-delta finding (deterministic), not an LLM guess.

@qq2514568380
qq2514568380 force-pushed the feat/graph-json-lines branch from df860db to 4b62160 Compare November 9, 2026 07:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Write graph.json one element per line (deterministic, line-friendly)

1 participant