Repository navigation
[sentry:7849148048] The load itself succeeded; the failure was the model-claim record write that follows it. src/core/sessions.ts:112 is await claim.update('ready'), reached only after ` - #115
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The load itself succeeded; the failure was the model-claim record write that follows it.
src/core/sessions.ts:112isawait claim.update('ready'), reached only afterruntime.load(modelId, …)resolved, and the frames place both the ENOENT cause and the IO_ERROR wrapper insidewriteClaimofsrc/lock/model-claim.ts(update→writeClaim→ tempwriteFile/rename, wrapped byclaimIoError). The claims folder and the per-model claim directory are created before the engine start, and nothing in the core removes a live claim directory (ModelClaimHandle.release()is the only remover and it is serialized against the load or awaited byreleaseAll()after the runtimes stop), so the directory was removed by something outside the claim protocol on that Windows host. The defect the event proves is the handling:writeClaimanswered any failure of the post-load record write as a failed load, andacquireNow's catch keeps the handle becauseruntime.findSession(modelId)is now true, so every later load or attach of the same resident model repeatedupdate('ready')on the same dead path and failed identically until unload. The repair insrc/lock/model-claim.tstreats ENOENT from the record write as a vanished directory: it re-createspathwithmkdir(path, { recursive: true })and writes the record once more; a second failure and every other errno keep the existingclaimIoError('write', …)wrapping, so a full volume, a file where the folder belongs or an unwritable folder still surface as coded IO_ERROR and still refuse. Probesrc/lock/model-claim-recovery.test.tswas mined on base (named assertionwrites the record again instead of failing the update) and now passes on the repaired code, together with its guard case that still expects a coded IO_ERROR. The controller's full-suite run reported one unrelated failure insrc/core/create.test.ts(wires the diffusion service to the control API, the journal, the events and the shutdown order—the abandoned job was not cancelledafter its own 10 s wait). That path is unreachable from this patch: the only added behaviour is the ENOENT branch ofwriteClaim, which runs solely when a claim-directory write fails with ENOENT, and the diffusion job cancellation neither creates nor deletes model claims. The same file documents in its header that whole-core tests failed at random under full-suite parallel load (T-282) and raises the per-test timeout to 20 s, while that inner 10 s poll loop is outside it — consistent with a load-induced flake rather than a regression.Evidence:
src/core/sessions.ts:96 acquires the claim before the engine start; line 112 is
await claim.update('ready'), the onlyupdatecall site inacquireNow, reached only afterruntime.loadresolved.src/lock/model-claim.ts
update→writeClaim→ tempwriteFile+rename, withclaimIoErrorproducingnew AtomicCoreError('IO_ERROR', …)— the message and code in the event.Event facts: error_code IO_ERROR, cause_code/diagnostic_cause_code ENOENT, operation load, component runtime, failure_reason unknown, release atomic-chat-core@0.23.4, os.name win32.
src/lock/model-claim.ts creates the claims folder recursively and the per-model directory with
mkdir(path)before writing the first record, so a later ENOENT means the directory disappeared between those calls and the post-load write.Only
ModelClaimHandle.release()removes a claim directory; its call sites are serialized bywithModelTransitionor awaited byreleaseAll()after runtimes stop, so an in-process deletion duringupdate('ready')is unreachable.src/core/sessions.ts keeps the claim handle when the failure happens after
runtime.findSession(modelId)is true, so the dead path poisoned every later attach of the resident model.Repaired
writeClaimre-creates the claim directory once on ENOENT and retries the record write; a second failure or any other errno staysclaimIoError('write', …).Probe src/lock/model-claim-recovery.test.ts now passes (2/2) in isolation: the vanished-directory case writes the record again with the same claim_id and state 'ready', and the claims-folder-replaced-by-a-file case still rejects with AtomicCoreError code IO_ERROR.
Controller full-suite output: 1 failed / 671 passed files, 11816 passed tests — the single failure is src/core/create.test.ts:1050, message 'the abandoned job was not cancelled'.
src/core/create.test.ts declares
vi.setConfig({ testTimeout: 20_000 })with a comment that these whole-core tests failed at random under the full-suite's parallel load (T-282), and the failing assertion is an inner 10 s poll loop, independent of the model-claim code.The added patch branch executes only when a claim record write fails with ENOENT; no test in the suite deletes a live claim directory except the new probe, so diffusion job cancellation sees byte-identical behaviour.
New ADR docs/decisions/2026-11-16-re-establish-a-model-claim-whose-directory-vanished.md documents the decision and is linked by the single bullet at docs/decisions/INDEX.md:261.
Regression tests: src/lock/model-claim-recovery.test.ts
Full CI and human review are required.
Report: http://127.0.0.1:8080/tenants/707d0855-80ab-4e1f-a156-f1c4546cbf52/runs/0a81fd6c-094f-4449-8e48-cb02f1b6c96e
Sentry: https://atomic-chat.sentry.io/issues/7849148048/