fix(server): prevent ResultManager from leaking mutable Task references.#480
fix(server): prevent ResultManager from leaking mutable Task references.#480leehpham wants to merge 1 commit into
Conversation
🧪 Code Coverage
Generated by coverage-comment.yml |
There was a problem hiding this comment.
Code Review
This pull request introduces a copy-on-write strategy and utilizes structuredClone within the ResultManager and DefaultRequestHandler to ensure that task snapshots remain immutable once they are handed to callers. These changes prevent background updates from mutating previously returned task references. The PR also adds comprehensive unit tests to verify this isolation. The review feedback identifies additional opportunities to strengthen this isolation by deep-cloning incoming status and artifact update events to prevent internal state from aliasing external objects that might still be modified by the executor.
| @@ -47,107 +49,75 @@ export class ResultManager { | |||
| await this.saveCurrentTask(); | |||
| } else if (event.kind === 'status-update') { | |||
| const updateEvent = event as TaskStatusUpdateEvent; | |||
There was a problem hiding this comment.
To fully isolate the ResultManager state from subsequent mutations by the executor and to avoid a potential race condition during the await at line 58, consider deep-cloning the updateEvent. Currently, next.status (line 68) aliases the status object from the event, meaning any later mutation of that object by the executor would be reflected in ResultManager's internal state and subsequent snapshots.
| const updateEvent = event as TaskStatusUpdateEvent; | |
| const updateEvent = structuredClone(event as TaskStatusUpdateEvent); |
References
- Use
structuredClonefor deep copying objects instead ofJSON.parse(JSON.stringify())to ensure thatundefinedfields are preserved.
| // If it's a final status update, the ExecutionEventQueue will stop. | ||
| // The final result will be the currentTask. | ||
| } else if (event.kind === 'artifact-update') { | ||
| const artifactEvent = event as TaskArtifactUpdateEvent; |
There was a problem hiding this comment.
Similar to the status update, the artifactEvent should be deep-cloned to ensure full isolation. Currently, the artifact objects are aliased into the artifacts array (lines 104 and 117), making the internal state susceptible to later mutations by the executor and race conditions during the await at line 89.
| const artifactEvent = event as TaskArtifactUpdateEvent; | |
| const artifactEvent = structuredClone(event as TaskArtifactUpdateEvent); |
References
- Use
structuredClonefor deep copying objects instead ofJSON.parse(JSON.stringify())to ensure thatundefinedfields are preserved.
Summary
Fixes #412.
ResultManagerwas mutatingthis.currentTaskin place on everystatus-updateandartifact-updateevent, and storing the task with a shallow copy on the initialtaskevent. The non-blockingsendMessagepath inDefaultRequestHandlerresolves the HTTP response promise withresultManager.getCurrentTask()— a live reference. Subsequent events then mutate the object the HTTP layer is about to serialize, so the client receives the post-execution state (e.g.failed) instead of the handoff snapshot (e.g.submitted).This is reference aliasing, not concurrency: the single
for awaitconsumer ofExecutionEventQueueprocesses events strictly in order. The handed-out reference and the in-flight state machine just happen to be the same object.Changes
src/server/result_manager.ts—processEventno longer mutatesthis.currentTask:taskingest:{ ...taskEvent }→structuredClone(taskEvent), so the executor'shistory/artifactsarrays are not aliased into ResultManager.status-update: builds a newTaskwith the updated status (and, if a message is present, a newhistoryarray). The previous reference is no longer touched.artifact-update: builds a newartifactsarray; when an existing artifact is being appended to, constructs a fresh artifact object rather than mutatingparts/metadatain place.src/server/request_handler/default_request_handler.ts—structuredClonethe result ofgetCurrentTask()at the non-blocking response boundary. Defense-in-depth: even if anyone later reintroduces internal aliasing inResultManager, the response handed to the caller is guaranteed to be a snapshot.Tests
test/server/result_manager.spec.ts(3 tests, this class previously had no direct test coverage):getCurrentTask()is unaffected by a subsequentstatus-update.getCurrentTask()is unaffected by a subsequentartifact-update.history/artifactsarrays passed in on the originaltaskevent are not mutated by later updates.test/server/default_request_handler.spec.tsmirroring the reproduction in [Bug]: ResultManager:processEvents (second events modifies first arriving event) which leads to unexpected behavior and race conditions #412: publishestask(submitted), advances timers, publishesstatus-update(failed), then re-asserts the originally-returned reference still showsstate: 'submitted'.All four tests fail on the prior implementation and pass after the fix. Full suite: 413/413 passing.
Behavior notes for reviewers
Taskvalue and then mutates its own copy expecting ResultManager to observe the mutation will no longer see that. This pattern is anti-spec (publish should have value semantics) and I'm treating it as a behavior fix rather than a regression. No tests or sample executors depended on it.artifact-updaterehydration path is now consistent with the in-memory path. The pre-existing rehydration branch (when nocurrentTaskwas in memory and the task was loaded from the store) only handledpartson append, silently droppingdescription/name/metadatamerges. The consolidated copy-on-write path applies the same merge logic in both branches.structuredClonerequirement. Already satisfied —package.jsonrequires Node ≥ 18, andstructuredClonehas been a Node global since 17.Test plan
npm test— full suite green (413 tests)npx tsc --noEmit— no new type errors in changed files (pre-existing errors insrc/samples/are unrelated to this change)npx eslinton changed files — cleantest/server/default_request_handler.spec.tsreproduces the issue when reverting the fix locally