Skip to content

Fix dynamic ignored-directory watcher replanning and degraded status - #973

Merged
7 commits merged into
tirth8205:stagingfrom
Bakul2006:watcher/replan
Nov 6, 2026
Merged

7 commits merged into
tirth8205:stagingfrom
Bakul2006:watcher/replan

Conversation

@Bakul2006

Copy link
Copy Markdown
Contributor

Pull Request

Linked issue

Closes #908

What & why

Fixes watcher planning when ignored directories are created or grow after the initial watch plan is installed.

Previously, the watch plan was calculated at startup. If an ignored directory such as node_modules/ or target/ did not exist at startup, its parent could receive a recursive watch. If the ignored directory was created later and grew beyond the existing split threshold, the watch plan was not reconsidered, allowing the ignored tree to fall under the recursive watch and potentially contribute to watcher/resource exhaustion.

This PR:

  • Marks the watch plan dirty when directory changes can affect the ignored-tree split.
  • Detects ignored-tree threshold crossings during shallow-watch synchronization.
  • Releases obsolete watches before installing the recalculated plan.
  • Preserves the existing watch adoption and budget behavior.
  • Marks startup budget fallback as degraded, so crg-daemon status correctly reports degraded coverage.

Regression tests were added for dynamically created/growing ignored directories and the startup degraded-status behavior.

How it was tested

The following validations were run successfully:

python -m py_compile code_review_graph/incremental.py
git diff --check

Language/static diagnostics reported no errors.

The full pytest/lint/type-check commands could not be run in the current environment because pytest, uv, and networkx are unavailable.

uv run pytest tests/ --tb=short -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional

These commands should be run in a fully provisioned development environment before merging.

Checklist

  • Tests added for new functionality
  • All tests pass: uv run pytest tests/ --tb=short -q
  • Linting passes: uv run ruff check code_review_graph/
  • Type checking passes: uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
  • Lines are at most 100 characters
  • Docs updated where behavior changed (README, docs/, docstrings)

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.55 (MEDIUM) — 31 changed function(s)/class(es), 0 affected flow(s), 12 test gap(s)

Risk-scored changes

Risk Level Symbol Location Tested
0.55 medium code_review_graph/incremental.py::_replan_if_requested code_review_graph/incremental.py:2166 no
0.40 medium code_review_graph/incremental.py::_plan_watch_paths_with_status code_review_graph/incremental.py:1976 no
0.35 low code_review_graph/incremental.py::request_replan code_review_graph/incremental.py:2150 yes
0.35 low code_review_graph/incremental.py::GraphUpdateHandler.dispatch code_review_graph/incremental.py:2810 yes
0.35 low tests/test_watch_robustness.py::TestRealObserver.Handler tests/test_watch_robustness.py:1241 no
0.30 low code_review_graph/incremental.py::_WatchSupervisor.__init__ code_review_graph/incremental.py:2070 yes
0.30 low code_review_graph/incremental.py::_replace_watch code_review_graph/incremental.py:2219 no
0.30 low code_review_graph/incremental.py::GraphUpdateHandler code_review_graph/incremental.py:2809 no
0.30 low tests/test_watch_robustness.py::TestNewDirectoryAdoption.FailingReplacementObserver tests/test_watch_robustness.py:402 yes
0.30 low tests/test_watch_robustness.py::FailingReplacementObserver.schedule tests/test_watch_robustness.py:403 no

Test gaps

  • code_review_graph/incremental.py::_plan_watch_paths_with_status (code_review_graph/incremental.py:1976)
  • code_review_graph/incremental.py::_remember_ignored_boundaries (code_review_graph/incremental.py:2154)
  • code_review_graph/incremental.py::_replan_if_requested (code_review_graph/incremental.py:2166)
  • code_review_graph/incremental.py::_replace_watch (code_review_graph/incremental.py:2219)
  • code_review_graph/incremental.py::GraphUpdateHandler (code_review_graph/incremental.py:2809)
  • ...and 7 more without direct tests

Token savings: this graph-backed report used ~43,019 fewer tokens (~89%) than reading every changed file in full (estimated, chars/4 approximation).


Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

@Bakul2006

Copy link
Copy Markdown
Contributor Author

Hey @tirth8205 just wanted to add on one note on validation: I added the regression tests, but I wasn't able to run the full pytest suite locally because the current environment is missing pytest, uv, and networkx.

The changes do pass py_compile, language diagnostics, and git diff --check.

Could you please run the full test suite in the project environment to verify the new watcher regression tests and the existing tests?

@Bakul2006 Bakul2006 changed the title Added the replanning structure Fix dynamic ignored-directory watcher replanning and degraded status Sep 11, 2026
@tirth8205

Copy link
Copy Markdown
Owner

Changes required: pending replans are never processed when the watcher has only a recursive root and no shallow watches. Run python -m pytest tests/test_watch_robustness.py -k 'new_large_ignored_tree_replaces_recursive_root_watch or ignored_tree_growth_replans_recursive_root'; both new regressions fail because _replan_if_requested is called only inside the empty shallow-watch loop. Process that request independently and retain existing coverage until replacement registration succeeds.

@Bakul2006

Copy link
Copy Markdown
Contributor Author

Hey @tirth8205 just did the same as you suggested me now the desired test are successfully passed

@tirth8205

Copy link
Copy Markdown
Owner

008051b fixes the original recursive-root replan tests, but existing source coverage is still lost when replacement registration raises OSError. Reproduce by growing node_modules beside src/deep/a.py and failing observer.schedule for src, or by removing the ignored tree and failing registration of a recursive root; after request_replan() and sync_watches(), later deep edits are no longer delivered. Four new preservation tests, including two real PollingObserver cases, pass main and the integration branch but fail this head. python -m mypy code_review_graph/ --ignore-missing-imports --no-strict-optional also reports five new Path/str errors in incremental.py, so the source changes still require rework.

@Bakul2006

Copy link
Copy Markdown
Contributor Author

Hey @tirth8205 just addressed the failed replan coverage regression.

@tirth8205

tirth8205 commented Sep 15, 2026 •

Copy link
Copy Markdown
Owner

This no longer merges into staging. Conflicts in code_review_graph/incremental.py.

git fetch origin && git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git push

When resolving:

  • Rebase the watcher/replan branch onto current origin/staging and resolve code_review_graph/incremental.py: staging (82ef9d0 'match ignore rules on repo-relative update inputs') changed sync_watches() to take an ignore_patterns kwarg and to compute newly_included via _should_ignore(...previous_patter...
  • After the rebase, re-run the full gate set (pytest 3.13, ruff, mypy) and re-check that tests/test_watch_robustness.py still exercises the replan + degraded-status paths against the merged sync_watches() semantics.

A large integration branch landed on staging today, which is why this drifted.

PRs now target staging, not main. Yours was retargeted already, so nothing to do there.

@tirth8205 tirth8205 added the needs-rebase Branch no longer merges into staging label Sep 15, 2026
@tirth8205

Copy link
Copy Markdown
Owner

This no longer merges into staging. The one conflict is code_review_graph/incremental.py, and the branch is 144 commits behind.

Worth knowing before you resolve:

  • code_review_graph/incremental.py: staging added a build-state checkpoint at the tail of full_build and moved the Git timeout into constants.py.
git fetch origin
git merge origin/staging
# resolve, then
uv run pytest tests/ -q
uv run ruff check code_review_graph/
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
git push

I have not reviewed the change itself yet. That comes once it merges and the checks run against the merged state, since staging has moved a long way and the result is what matters.

@tirth8205

Copy link
Copy Markdown
Owner

Thank you for this PR.

I tested your branch, and a copy with the conflict markers removed.

Before I can merge it, please fix these:

  1. code_review_graph/incremental.py has merge conflict markers at lines 2149 and 2307. Python cannot load the file. import code_review_graph.incremental fails with SyntaxError at line 2149. CI fails because of this.
  2. degraded never goes back to False after a replan. _replan_if_requested only sets it to True (incremental.py:2215). Example: set max_schedules=2 and add three big node_modules folders. Delete them, then replan. The new plan is exact, but degraded stays True.
  3. Nothing calls _replace_watch (incremental.py:2219). Please use it or remove it.

Please update your branch with the latest staging. There are conflicts in code_review_graph/incremental.py. Please keep staging's rewatch_all(). Also keep the _prune_unwatched() call in sync_watches().

After you push the fix, I will test it again.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved merge-conflict markers in incremental.py make it invalid Python, preventing watch mode and tests that import the module from running.

4 open issues
What changed in this PR

This PR addresses #908 by updating watcher plans as ignored directories appear or grow, and reporting startup budget fallback as degraded coverage.

Changes:

  • Requests replanning on directory events and detects ignored-tree threshold crossings during synchronization.
  • Adds replacement-watch registration and rollback handling to preserve coverage when replanning fails.
  • Adds regression tests for dynamic ignored trees, failed replans, and startup health reporting.
File Description
tests/​test_watch_robustness.py Adds replanning, failure-recovery, and degraded-status regression tests.
code_review_graph/​incremental.py Adds dynamic watch replanning, rollback handling, and startup fallback status.

Review effort: Balanced

else:
self._shallow.add(key)

<<<<<<< HEAD
Comment on lines +2170 to +2174
plan, fallback = _plan_watch_paths_with_status(
self._repo_root,
self._ignore_patterns,
max_schedules=self._max_schedules,
)
f"rollback watch registration {key}",
timeout=_WATCH_STOP_TIMEOUT,
)
return
Comment on lines +2215 to +2216
if fallback:
self._degraded = True
@tirth8205 tirth8205 closed this pull request by merging all changes into tirth8205:staging in 0160eb0 Nov 6, 2026
gowrishacv pushed a commit to gowrishacv/code-review-graph that referenced this pull request Nov 6, 2026
…ew of tirth8205#973)

A failed replan now stays pending and retries after a cool-down. A successful one
clears degraded status and any refusals it covers. When no plan fits the budget,
the watches in place stay rather than a recursive root taking over. Removes the
conflict markers and the unused _replace_watch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gowrishacv pushed a commit to gowrishacv/code-review-graph that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase Branch no longer merges into staging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

watch: ignored trees created after startup are watched recursively, re-creating #811's exposure

3 participants