Repository navigation
A stalled job is named while it stalls, ends libc_children, and a suite the driver ends leaves no scratch - #1229
Conversation
…te the driver ends leaves no scratch `libc_children` runs 23 programs in one boot. A program that hung was named nowhere: the row printed failures only after its last program, each later program then spent its own whole guard in silence because the guest's runner was still waiting for the hung job, and the driver's 900 s silence guard ended the suite first, whose killed processes left their scratch for a second red (guest-rows run 42949792797). - `run_test` says once a minute which job it waits for, how far into its guard it is and when the guest was last heard. Every row waits for its jobs there, so a stalled job is named while it stalls in any of them, and the driver hears a harness at a bounded wait rather than a hung one: its guard no longer outruns a job's scaled ceiling. - `libc_children` says each program's verdict as it ends, failures included, and ends at the first program with no exit code, naming it and the programs not run. It was the one row that went on after such a job; the others return at their first failing job. - The driver removes what the processes of a group it ended left under the job's `$TMPDIR` and `/tmp` before it judges what is left (`toyos_tmpdir::reclaim_gone`, by the sweep's own protocol). The hung step is the red; its scratch no longer stays on the host until a later process sweeps, nor reds a second step. Closes issues/a-program-that-stalls-in-libc-children-is-not-named-and-the-driver-ends-the-suite.md. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Review of #1229 at 96514f2ArchitectureThe change has three parts, and each sits with the right owner:
The change adds about 75 production lines and 80 test lines. It leaves one duplicate behind (B2). FindingsBLOCKER B1 —
BLOCKER B2 —
SUGGESTION (optional):
Correctness (verified)
Is the claim held by a test?
Not exercised end to end
None of these needs a new guest run. SEND BACK (B1, B2). Both fixes are host-only: |
…adopt goes with the job reclaim_gone took The ended-group test's "red while they stand" passed on the private directory's missing lock, before `left_behind` looked at a root. It now writes the lock as its siblings do and holds the refusal to naming the root under each base, each in its own clause; a `left_behind` blind to either base fails it. `nothing_left` reads whether the steps wrote to `$TMPDIR` before the reclaim, which makes that lock itself, so a driver that ended a group still refuses a `$TMPDIR` the steps never saw. Its line says the roots it removed were of gone processes, which is what it takes. `TempDir::adopt` had no caller and did for one pid what `reclaim_gone` does for every gone root; it and its test are deleted. `judge`'s header says that any program with no exit code ends the row, a refused spawn among them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts named before the reclaim, two mutations, toyos-tmpdir without adopt
|
Round 2 at 8841591 (one commit on 96514f2):
Checks: |
Re-review of #1229 at 8841591 (round 2)ArchitectureThe architecture is the same as in round 1. This round deletes the duplicate mechanism and makes the test meaningful.
Previous blockers
Suggestions taken (checked)
Evidence
PR text (for the orchestrator)The full-host-check bullet calls this round's FindingsNone. LAND |
|
Orchestrator: follow-up review r2 LAND at 8841591 (B1 and B2 resolved; host-only change, evidence at a clean tree). I corrected the body's size of this round's |
Problem
libc_childrenruns 23 programs in one boot, onerun_testeach. When a program hung, nothing named it: the row printed failures only after its last program, each later program spent its own whole guard in silence because the guest's runner was still waiting for the hung job, and the driver's 900 s silence guard ended the suite first. The killed processes' scratch then went red as a second step and stayed in/tmpon the runner (guest-rows run 42949792797). Recorded inissues/a-program-that-stalls-in-libc-children-is-not-named-and-the-driver-ends-the-suite.md, which this closes.Resulting behaviour
QemuInstance::run_testsays once a minute which job it waits for, how far into its guard it is and when the guest was last heard. Every QEMU row waits for its jobs there, so this holds for all of them, and the driver hears a harness at a bounded wait: its silence guard no longer outruns a job's host-scaled ceiling (a 300 s guard is up to 2400 s at the scale's cap).libc_childrensays each program's verdict as it ends, and ends at the first program with no exit code, naming it and the programs not run. A stalled program costs one guard.$TMPDIRand/tmpbefore it judges what is left (toyos_tmpdir::reclaim_gone, by the sweep's own protocol: moved under the lock into a root of the driver's there, removed outside it). Whether the steps wrote to$TMPDIRat all is read before that removal, which makes the lock it is read from. The hung step is the one red.TempDir::adopt, which did this for one pid and had no caller, is deleted.Decisions
run_testis what names it when it happens.run_testcaller intests/toyos.rsandtests/common/returns at its first failing job (cryptojobs,job_windowsequences, the designation pair), so none goes on after a stalled one. They share only the silence, which (1) covers. No separate record.reclaim_gonegoes through a root of the caller's rather than deleting under the lock:/tmpis shared by every process of the user, and the lock is held for renames only.$TMPDIR, including one of a process that died by itself in another step of the same job; that step is red on its own account, and the step's line says the roots were of gone processes.libc_children, a spawn the runner refused (error=) among them, as a failing job ends every other row;judge's header now says so rather than giving only the hung job's reason.Checks
cargo test --test toyos-checks a_stalled_program: exit 0.checks::a_stalled_program_ends_its_row_and_is_namedfeeds the row's judge a passing program, a failing one and the result a guest silent past its guard gives (ceiling_verdict's own sentence in aWaitVerdict), and asserts the later programs were never asked for, the red names the program and those not run, and the suite files it as a blown guard.break, that test fails on the programs asked for (exit 101).cargo test -p toyos-tmpdirat 8841591: exit 0, 13 passed,adopt's test gone (h-tmpdir.log); newa_gone_root_is_reclaimed_when_asked_and_a_live_one_never(aSIGKILLed holder's root goes, a live one, an unasked one and the lock stay).cargo test --lib ci::testsat 8841591: exit 0, 21 passed (h-ci.log).what_an_ended_group_left_is_removed_and_an_earlier_root_stayswrites the private directory's lock first, as its siblings do, and holds the refusal before the reclaim to naming the root under$TMPDIRin the leftovers' clause and the one under/tmpin the dead-process clause; after the reclaim both are gone and the root dead before the steps stays.left_behindignoring the private directory's roots (h-blind.log, patch), and ignoring the dead roots under/tmp(h-blind-short.log, patch). Before this round the test passed with the private directory's roots ignored: its refusal was the missing lock.cargo test --test toyos-checks a_stalled_programat 8841591: exit 0 (h-stalled.log).cargo clippy -p toyos-tmpdir -p toyos-build --all-targetswith the adopted lints at 8841591: exit 0 (h-clippy.log).libc_childrenat 96514f2: green, run 43022077634, 8 s of testing.stalled-program-named-control(libc_read_restartshangs in silence, bound 50 s): run 43022082892, red as intended in 86 s. At 60 s[qemu] test_rs_libc_read_restarts has run 60s of its 78s guard, the guest last heard 30s ago; at the guardlibc_read_restarts: FAILED, it ended with no exit code; 1 program(s) after it were not run: tty_interrupt;STALL libc_children;nothing left in $TMPDIR or /tmpgreen; one red step. Bounded before dispatch: the host scale is at most 8, so the guard expires by 50 * 8 + 15 = 415 s, and with both the report and the stop broken the two waits are 830 s, inside the driver's 900 s.cargo run -- --ci hostat 96514f2, before this round's test, comment and roughly 20-lineci.rschange, which the checks above cover (macOS arm64,harness-ci-host-interrupted.log), the change touching the harness and the build system'sci.rs:llvm::tests(385 passed, 1 failed), the known macOS defectissues/the-llvm-environment-test-breaks-git-on-macos.md; the newci::testspass;toyos-tmpdiramong them),toyos-cryptoin its three forms, the kernel's library and the licences: green;target/was removed under them, and the userland step was not reached. Neither covers this change, which touches no kernel code and no program undersystem/,apps/ortests/guest/; by the owner's testing policy they are not owed, and the merge queue runs the full host check.Evidence: ToyOSOrg/evidence
pr-1229; this round at 0a1be3a9 (linked above), the first at a2af643c: the guest runs (guest-green-43022077634.log,guest-control-43022082892.log) and the control's diff (guest-control.patch); the host tests and their control (checks-test-1.log,ci-test-1.log,tmpdir-test.log,harness-control.log,harness-control.patch); the full host check above; and the run that named the defect (run-42949792797.log).Risks and what is not covered
ENDED_A_GROUP, set where the group is signalled) is read in review. No guest control exercised it: one would hold the runner for 900 s.run_test(await_guest,drain_until,await_exit) still say nothing while they wait. Their bounds are 120 s or less before scaling, so only a host at the scale's cap brings one near the driver's guard; not observed.🤖 Generated with Claude Code