[wgu9]

Partially addresses #436. JSON support remains out of scope for this PR.

What changed

Generic .yml / .yaml files now produce value-free, path-addressable graph
structure:

• aggregated YamlPath nodes for mappings and repeated sequence shapes
• occurrence counts plus bounded source-line/range samples
• mapping-scoped duplicate-key diagnostics with exact line samples
• canonical CONTAINS edges and REFERENCES edges for aliases/merge keys
• distinct reference targets for typed and duplicate anchored definitions
• changed_yaml_paths in change/review output
• CLI kind filtering, safe graph serialization, visualization, and docs support

Scalar values are deliberately discarded. The graph stores structural key paths
and YAML types, not configuration values. Query, search, and change-analysis
responses share the same nested yaml metadata contract.

Safety and compatibility

This incorporates the earlier YAML review feedback from #539, #577, and #462:

• malformed or pathologically recursive YAML fails safely
• aliases, root/self aliases, merge sequences, and complex mapping keys have
  regression coverage
• per-file limits bound node count, nesting depth, exact/schema/identity path
  length, and diagnostic samples
• repeated sequence shapes aggregate under [*] paths
• existing Ansible and Spring parsers retain precedence
• files rejected by a specialized path heuristic fall back to generic YAML
• full-build/query coverage asserts zero dangling YAML edges

How this differs from earlier attempts

Earlier review concern	This PR
Bare or dangling graph endpoints	Canonical persisted identities plus full-build dangling-edge assertions
Raw configuration values in metadata	Values are never stored
Framework- or organization-specific YAML schema	Generic structural paths and YAML types
One node per repeated list item	Aggregated [*] schema paths with bounded occurrence samples
Missing malformed/cycle/size handling	Error, recursion, node, depth, path, and metadata bounds
Parser-only coverage	Persistence, query, change analysis, visualization, and specialization regressions

Verification

Verification uses only synthetic fixtures and this public project's test suite:

• Full suite: 2,413 passed, 7 skipped, 2 xpassed
• Focused YAML/change/incremental suite: 158 passed
• Ruff (source and changed tests): clean
• Mypy (same flags as CI): clean
• git diff --check: clean

uv run --python 3.13 pytest tests/ --tb=short -q
uv run pytest tests/test_yaml_parser.py tests/test_changes.py \
  tests/test_incremental.py -q --tb=short
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional

The synthetic coverage includes persisted graph queries, zero dangling YAML
edges, scalar-value absence, malformed and recursive input, oversized paths,
duplicate keys, aliases, merges, multi-document YAML, and Spring/Ansible
precedence.

Deliberate non-goals

This first phase does not add JSON indexing, scalar-value indexing,
duplicate-value detection, cross-file business-rule validation, business-owner
inference, or Helm values-to-template resolution (#199). Those require explicit
policy and privacy semantics on top of structural indexing.

[tirth8205]

Solid implementation and genuinely good coverage. Two things before I take it.

1. Large generated YAML has no gate. pnpm-lock.yaml is plain YAML and is not in DEFAULT_IGNORE_PATTERNS (incremental.py:200 has package-lock.json, yarn.lock, *.lock, but not pnpm). A synthetic 6.7MB lockfile takes about 15s in _parse_yaml and fills the whole 10k node cap with lockfile noise in search and stats, and that cost hits the update hook on every dependency bump. Add pnpm-lock.yaml to the default ignores plus a byte-size cutoff for generic YAML.

2. _parse_yaml parses everything twice: compose_all plus a full yaml.parse event pass just to collect anchors, roughly doubling CPU per file (7.9s compose vs 9.0s events on that lockfile). Skipping the event pass when the source contains no '&' would cover almost all real files.

Minor, fine to leave: anchors under unsupported complex keys mislabel the first alias site as anchor_definition and shift later REFERENCES line attribution. Sanitization, ansible/spring precedence, and the dangling-edge guarantees all held up under adversarial testing. No DB migration needed since kind is unconstrained TEXT.

[wgu9]

Thanks, I fixed both items and merged the latest main in 76e8352.

Generic YAML now stops at a 1 MiB byte limit before decoding or calling PyYAML. pnpm-lock.yaml is also ignored by default. Files without an ampersand skip the second event parse, while the existing anchor tests still cover the event path.

Verification:

• 3,004 passed, 9 skipped, 2 xpassed
• Ruff passed
• mypy passed
• git diff --check passed

[tirth8205]

Outside this sweep's mission of demonstrated coding-session token savings without context loss: generic YAML indexing adds a new node contract and changes parsing, diagnostics and visualisation across 20 files. The improved bounds are useful, but a representative coding-task comparison is still needed to justify that new surface in this work.

[tirth8205]

This no longer merges into staging. Conflicts in README.md, code_review_graph/docs/LLM-OPTIMIZED-REFERENCE.md, docs/COMMANDS.md, docs/FEATURES.md, docs/USAGE.md, docs/schema.md.

git fetch origin && git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git push

When resolving:

• Rebase PR onto current origin/staging and resolve conflicts in README.md, code_review_graph/docs/LLM-OPTIMIZED-REFERENCE.md, docs/COMMANDS.md, docs/FEATURES.md, docs/USAGE.md, docs/schema.md
• Re-run pytest/ruff/mypy and the edge-case review after the rebase

A large integration branch landed on staging today, which is why this drifted.

PRs now target staging, not main. Yours was retargeted already, so nothing to do there.

[tirth8205]

This no longer merges into staging. 8 files conflict and the branch is 226 commits behind:

• README.md
• code_review_graph/changes.py
• code_review_graph/docs/LLM-OPTIMIZED-REFERENCE.md
• code_review_graph/parser.py
• docs/COMMANDS.md
• docs/FEATURES.md
• docs/USAGE.md
• docs/schema.md

Worth knowing before you resolve:

• code_review_graph/changes.py: staging moved _GIT_TIMEOUT into constants.py as GIT_TIMEOUT and added a separate discovery budget.
• code_review_graph/parser.py: staging replaced _extract_import with _extract_import_specs returning _ImportSpec(module, line, form), and every IMPORTS_FROM edge now carries extra={IMPORT_SCOPE_KEY: scope}. If your branch also writes to extra, merge the dicts rather than taking one side.

git fetch origin
git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git push

I have not reviewed the change itself yet. That comes once it merges and the checks run against the merged state, since staging has moved a long way and the result is what matters.

[tirth8205]

Thank you for this PR. Your generic YAML work is very useful.

This change is already in #1092. That PR has both of your commits on top of the latest staging. Your name stays on those commits, so the credit is yours. I ran the YAML tests on both branches. I also parsed the same YAML files on both. The results were the same. #1092 merges cleanly. This branch now has conflicts with staging.

So I will not merge this one. I will use #1092 instead. #1092 needs two small fixes first. I asked for them there. You do not need to do anything more here.

Thank you again for your help.

[Copilot]

🟡 Changes recommended

Shared-alias traversal can grow exponentially and stall graph builds despite emission limits, while additional defects affect alias source tracking, key semantics, and YAML search metadata.

6 open issues

• Track shared alias targets to prevent exponential traversal · New
• Include YAML metadata in semantic search results · New
• Advance alias-line accounting when skipping long keys · New
• Canonicalize scalar keys before duplicate detection · New
• Detect merge keys by YAML tag, not key text · New
• Use source marks for aliased mapping keys · New

What changed in this PR

Adds value-free generic YAML indexing to the code graph, partially addressing #436 while leaving JSON support out of scope.

Changes:

• Introduces aggregated YamlPath nodes, duplicate-key diagnostics, and alias references while preserving Ansible and Spring precedence.
• Adds YAML-aware change analysis, serialization, CLI filtering, and visualization.
• Expands regression coverage and documents YAML indexing behavior.

File	Description
tests/​test_yaml_parser.py	Tests structural indexing, bounds, aliases, persistence, and queries.
tests/​test_visualization.py	Checks YAML node and reference-edge visuals.
tests/​test_spring_config.py	Verifies generic YAML fallback outside Spring configuration.
tests/​test_incremental.py	Checks the pnpm lockfile ignore pattern.
tests/​test_changes.py	Tests YAML occurrence-range change mapping.
tests/​test_ansible_parser.py	Verifies generic fallback for non-Ansible YAML.
README.md	Adds YAML indexing to the feature overview.
docs/​USAGE.md	Documents YAML support and node kinds.
docs/​schema.md	Defines YAML metadata and graph identities.
docs/​FEATURES.md	Updates parser and change-analysis capabilities.
docs/​COMMANDS.md	Documents YAML filters and change output.
code_review_graph/​visualization.py	Adds YAML styling and reference-edge controls.
code_review_graph/​tools/​review.py	Exposes YAML change results and minimal-output counts.
code_review_graph/​parser.py	Implements generic YAML extraction and specialized fallbacks.
code_review_graph/​main.py	Updates tool docstrings for YAML support.
code_review_graph/​incremental.py	Excludes pnpm lockfiles by default.
code_review_graph/​graph.py	Adds sanitized nested YAML metadata serialization.
code_review_graph/​docs/​LLM-OPTIMIZED-REFERENCE.md	Updates the bundled language-support reference.
code_review_graph/​cli.py	Allows filtering searches by YamlPath.
code_review_graph/​changes.py	Maps changed YAML ranges and returns bounded path results.

_{Review effort: Balanced}