Skip to content

[Bug]: --update merge block overwrites .graphify_extract.json, so re-running it or Step 9 stamps a doc whose chunk failed #4811

Open

Description

@gitmotion

Pre-flight checks

  • I have checked the Troubleshooting section in the README

What happened?

The --update merge block reads this run's extraction from graphify-out/.graphify_extract.json (update.md:104), stamps the manifest from it, and then overwrites the same file with the whole merged graph so Step 4 sees it (update.md:132). After that, anything that reads .graphify_extract.json as "what this run produced" sees every doc's old nodes as fresh output:

Either way the doc is marked done, so it is not re-queued, and the protection from #2015 and #1948 is lost. Its graph keeps the nodes from before the edit.

Expected: after an --update in which a changed doc's chunk failed, the doc stays queued after a second run of the merge block and after Step 9.

Steps to reproduce

The script renders the Claude skill with tools.skillgen, runs its real --update blocks through bash on a one-doc project where the changed doc's chunk failed, then runs either the merge block again or Step 9. Run it from the root of a checkout.

python merge_rerun_repro.py rerun
python merge_rerun_repro.py step9
merge_rerun_repro.py
"""After an --update in which a changed doc's chunk failed, a re-run of the merge block, or Step 9,
stamps the doc as done. Run from the root of a graphify checkout:

    python merge_rerun_repro.py rerun   # run the update merge block a second time
    python merge_rerun_repro.py step9   # run core Step 9 after the update

Renders the Claude skill with tools.skillgen and runs its real blocks through bash.
"""

import json
import os
import re
import subprocess
import sys
import tempfile
from pathlib import Path

REPO = Path.cwd()
sys.path.insert(0, str(REPO))
from tools.skillgen import gen  # noqa: E402

rendered = {a.path: a.content for a in gen.render_all(gen.load_platforms(), only="claude")}
update = re.findall(r"```bash\n(.*?)```", rendered["graphify/skills/claude/references/update.md"], re.S)
core = re.findall(r"```bash\n(.*?)```", rendered["graphify/skill.md"], re.S)
pick = lambda blocks, anchor: next(b for b in blocks if anchor in b)  # noqa: E731
detect = pick(update, "result = detect_incremental(")
populate = pick(update, "'all_files': r.get('files'")
merge = pick(update, "build_merge(")
step9 = pick(core, "Update cumulative cost tracker")

env = {**os.environ, "PYTHONPATH": str(REPO), "GRAPHIFY_OUT": "graphify-out"}
proj = Path(tempfile.mkdtemp())
out = proj / "graphify-out"
out.mkdir()
(out / ".graphify_python").write_text(sys.executable)


def py(source: str) -> str:
    return subprocess.run([sys.executable, "-c", source], cwd=proj, env=env, check=True,
                          capture_output=True, text=True).stdout


def bash(block: str) -> None:
    block = block.replace("INPUT_PATH", ".").replace("IS_DIRECTED", "False")
    subprocess.run(["bash", "-c", block], cwd=proj, env=env, check=True, capture_output=True)


def queued() -> list[str]:
    return json.loads(py("import json\nfrom pathlib import Path\nfrom graphify.detect import detect_incremental\n"
                         "print(json.dumps(sorted(Path(f).name for fl in detect_incremental(Path('.'))"
                         "['new_files'].values() for f in fl)))"))


# A built project: notes.md extracted and stamped, its node in graph.json.
(proj / "notes.md").write_text("# Notes\nWidget\n")
py("from pathlib import Path\nfrom graphify.cli import _stamped_manifest_files\n"
   "from graphify.detect import detect, save_manifest\n"
   "x = {'nodes': [{'id': 'notes_widget', 'source_file': 'notes.md'}], 'edges': [], 'hyperedges': []}\n"
   "save_manifest(_stamped_manifest_files(detect(Path('.'))['files'], x, Path('.')), root='.')")
(out / "graph.json").write_text(json.dumps({"nodes": [{"id": "notes_widget", "label": "Widget",
                                                       "source_file": "notes.md"}], "edges": [], "hyperedges": []}))

# --update: notes.md changed and its chunk failed, so this run's extraction has nothing for it.
(proj / "notes.md").write_text("# Notes\nWidget and Sprocket\n")
bash(detect)
bash(populate)
(out / ".graphify_extract.json").write_text(json.dumps({"nodes": [], "edges": [], "hyperedges": [],
                                                        "input_tokens": 0, "output_tokens": 0}))
bash(merge)
print("after the update's merge block, next --update queues:", queued())

bash(merge if sys.argv[1] == "rerun" else step9)
print(f"after {'a second merge' if sys.argv[1] == 'rerun' else 'Step 9'}, next --update queues:", queued())

Error output or graph output

== rerun
after the update's merge block, next --update queues: ['notes.md']
after a second merge, next --update queues: []
== step9
after the update's merge block, next --update queues: ['notes.md']
after Step 9, next --update queues: []

Graphify version

v8 at dac0634 (0.9.96)

Operating System

macOS

Python Version

3.12

Installation Method

built from source (git clone)

Additional Environment Details

No provider API keys set; the script calls no LLM.

Additional context

Fix sketch. Stamp from a file the merge does not overwrite. Part B already leaves this run's semantic result in .graphify_semantic.json (cache hits included), and code files are stamped regardless of output, so the merge block and Step 9 could both pass that to _stamped_manifest_files. That is the fix #2844 proposes for Step 9; it would cover both paths here. Alternatively, the merge block could keep this run's extraction under its own name before overwriting .graphify_extract.json.

Related: #2844 (Step 9 stamps from the AST + semantic merge on a full build), #2865 (same block: stamps after build_merge mutated its input), #4807 (partly-read docs on the same path).

Activity

  1. github-actions commented on Oct 27, 2026

    @github-actions

    Thanks for opening this issue, @gitmotion. A maintainer will take a look soon.

    If you would like to discuss it in real time, come say hi on our Discord server. For longer-form questions and ideas there is also GitHub Discussions.

    To help us triage, please make sure the report includes what you expected, what actually happened, and the steps (and a small sample) to reproduce it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions