Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ itself on its own hardware and reproduces, byte for byte, what the host built.
| ✅ | The whole OS booted across many machine shapes, in parallel, as an ordinary `cargo test` |
| ✅ | An audio gate that compares captured device output against a recorded baseline, statistically |
| ✅ | Kernel decisions in plain crates, tested on the host in milliseconds |
| ✅ | Reading a machine with no serial port — panics, logs and a blocked-task dump painted on its own panel |
| ✅ | Reading a machine with no serial port — panics and logs painted on its own panel |
| ✅ | LLDB against a running kernel |
| ⬜ | A Windows host |

Expand Down
24 changes: 24 additions & 0 deletions issues/a-counters-read-under-host-load-can-go-silent-for-15-s.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,30 @@ the step each stopped in is not read from it:
77 (38 s, with ceilings paid at 8.00x and 31 s between its image and its
first guest line) and 75 to 67 (7 s).

Three in one run, `wt/toyos-nohotkey` at `22b7bb464`, the whole suite, 41 of
44: `virt_smp` (`waiting for the job test_rs_counters_read to end`),
`virt_mask_windows` and `virt_off_names_the_cpus_left_on` (each `waiting for
the boot's last word`), all three verdicts within 4 s of the host's clock.
Load 54.08 60.22 55.97 when the suite began and 71.66 62.30 57.69 when it
ended, 14 cores; fastest boot 716 ms against the reference 1424 ms, ceilings
at 1.00x; workers 1309 s building against 1181 s testing. `virt_smp`'s last
three lines are this file's profile: `===TEST_END unmap_touch exit=0===` and
`===TEST_START test_rs_counters_read===` at 11.606 and 11.607, the spawn of
`pid=13` at 11.616 on cpu4. The other two said nothing while they were waited
on, so the step each stopped in is not read; `virt_off_names_the_cpus_left_on`
is also one of `issues/the-boots-last-word-can-miss-the-console-when-klogd-holds-the-wire.md`'s
waits. The branch removes the blocked-task dump from `drain_irqs`, which every
pass runs. The three alone at that commit passed (load 56.98 to 54.99). Not
known: whether the guests ran; no register capture was taken. Against that
removal as the cause: at `2375834a6`, which already carries it and whose
guest kernel differs from `22b7bb464`'s only by three `pub` methods with no
caller, the whole suite passed 43 of 43, the same three among them, at load
28.66 rising to 61.52; what it took out of a pass is three unarmed checks
that each load an atomic and return; and the read is answered from the kick
vector (`counters::serve_here`), not from a pass. The three images were
built within 2 s of each other, after a 23 s wait on the artifact lock, and
the three guests failed together.

The fork compiler's ScalarEvolution fault (llvm/llvm-project#175729, which
`src/miscompile.rs` now refuses a sysroot for) is not their cause.
`tests/virtsmpcase`'s image was built and `virt_el1_smp` run by the compiler
Expand Down
3 changes: 1 addition & 2 deletions issues/a-double-fault-on-cpu-1-under-a-wide-suite.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,7 @@ to a failure arm reaches it: boot it with `BootOptions { qmp: true, .. }` so the
guest survives the verdict, and ask `human-monitor-command` for
`info registers -a` over the lane's socket to see what the *other* CPU was
doing. Take that capture before injecting anything — a keystroke revives a
halted CPU, so Ctrl+Alt+D both confirms the diagnosis and destroys the evidence
for it.
halted CPU, so input over the same socket destroys the evidence for it.

## What the surviving line establishes about the machine

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ opened: 2026-10-03
`panic::halt_all_cpus` says the machine holds its report "with interrupts
masked" and masks nothing itself: the panic handler and the exception entries
mask before they reach it. Two callers do not, both in the test kernel alone:
the `panel-painter-stalls` actuator, from a scheduler pass
(`kernel/src/drivers/panic_console/mod.rs`'s `stall`), and `SYS_DEBUG`'s
the `panel-painter-stalls` actuator, from the boot thread after the last boot
phase (`kernel/src/drivers/panic_console/mod.rs`'s `stall`), and `SYS_DEBUG`'s
`FATAL_HALT` (`kernel/src/syscall/dispatch.rs`). The CPU that then holds the
panel spins on the reset bound with `IF` set, and every device handler pinned
to it still runs on a kernel that has declared itself dead.
Expand All @@ -22,7 +22,9 @@ and the halted one's has it clear. With `hold_the_panel` patched to poll port
0x60 and a key pressed inside the bound, six boots: the patched poll read the
key in four, each with the panel's CPU cpu1, and never saw it in two. The one
of those two whose console was kept had the panel on cpu0, where the i8042's
handler is pinned.
handler is pinned. Those boots staged the actuator from the Ctrl+Alt+D
report's painter, in a scheduler pass; that hotkey is removed, and the reading
has not been taken since the actuator moved to the boot thread.

So that test stages a fatal path no shipping kernel takes. Not measured: that
the handler read the byte in the two boots the poll missed it; the open `IF`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ would leave a whole boot idle straight after a sibling thread's clean exit.
path's two posts.

**Exit condition.** A capture taken from a boot that has actually stopped, which
names the first waiter and the subject it waits on — the blocked-task dump, or
that boot's trace diary and panic records.
names the first waiter and the subject it waits on — that boot's trace diary
and panic records.

**The periodic reporter is gone.** The owner ruled on 2026-10-04, choosing
"Remove it entirely": "Delete the periodic report and its counters; hang
Expand Down
2 changes: 1 addition & 1 deletion issues/a-stop-shows-nothing-on-the-panel.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ opened: 2026-10-04

A shutdown or reboot a press or a program asks for puts nothing on the
screen: the kernel's panel console (`kernel/src/drivers/panic_console/`) paints
only a panic and the Ctrl+Alt+D report, and on a machine without a compositor,
only boot progress and a panic, and on a machine without a compositor,
which is every test image, nothing else paints at all. On the T14 the owner
pressed the power button and saw nothing happen, which is also what a machine
that ignored the press looks like.
Expand Down
3 changes: 1 addition & 2 deletions issues/ap-tsc-trail-is-assumed-and-never-checked.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,7 @@ zero instead of a fault.
record it writes with zero for as long as it trails, and those records read as
the oldest thing the machine has: `log/read.rs`'s merge sorts them last, and its
per-shard descent — which relies on `at_ns` descending with the sequence number
— stops at the first of them. A bracketed report (Ctrl+Alt+D) would then carry
nothing from that CPU. Every other consumer of `nanos_since_boot` on that CPU is
— stops at the first of them. Every other consumer of `nanos_since_boot` on that CPU is
wrong by the same amount: deadlines, the scheduler's accounting, `at_ns` in
`/log`.

Expand Down
140 changes: 0 additions & 140 deletions issues/blocked-dump-cannot-fire-on-a-total-freeze.md

This file was deleted.

17 changes: 2 additions & 15 deletions issues/blocked-time-is-invisible-while-the-park-lasts.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ is charged to (`WaitClass` is the wait's now, named at the arm), not when.

The fix has the same shape as `cpu_ns`'s and is not free: a reader would need
the park's `since` stamp and its class from the owning CPU, and a `CpuSched` is
`!Sync` — `sched::dump`'s `for_each_parked` reaches only the calling CPU.
`!Sync` and reachable from its own CPU alone.
`TaskHandle` is the cross-CPU face and could publish `(class, since)` at the
park the way it publishes `running_since` at the dispatch, which is two relaxed
stores on the park path. Whether that is worth two stores per park is the
Expand All @@ -47,13 +47,7 @@ parked thread's blocked time is invisible until the park ends, exactly
backwards for reading a wedge in progress. A scoped fix is named. Owed to
whoever next extends the diagnostics the T14 wedge investigation started.

## Two corrections, 2026-09-01

**The Ctrl+Alt+D report already answers this and is not the site.** `report_this_cpu`
prints `task.class.name()` and `Ms(now.saturating_sub(task.since))` for a task
that is still parked (`kernel/src/sched/dump.rs:376-381`), reading
`ParkedInfo::since` — "When the park began" (`kernel/src/sched/driver.rs:830`).
The live interval and its class are both there.
## A correction, 2026-09-01

**`SYS_PROCESS_STATS` misses more than the park in progress.**
`ProcessData::accounting`'s five `blocked_*_ns` fields are written by
Expand All @@ -67,10 +61,3 @@ threads' parks only: every park a still-live thread has already finished is
invisible too, not merely the one it is in. Publishing `(class, since)` at the
park closes the smaller half; the larger half is a live thread's completed
parks reaching a cross-CPU reader at all.

**And the dump is not a substitute for the counters**, so "already answers this"
above is about the *site*, not about the need. It paints the panel on
Ctrl+Alt+D, never answers `SYS_PROCESS_STATS`; it prints at most
`LINES_PER_CPU` = 16 ordinary parked lines per CPU (`kernel/src/sched/dump.rs:32`,
truncating at `:370`) and counts the rest; and it reports a park's duration, not
the per-class totals `ps` reads.
27 changes: 17 additions & 10 deletions issues/desktop-window-child-freeze.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,10 @@ machine that is still running passes somewhere. Judge the next occurrence by the
signature exactly as before; a green run of this test proves what it always
proved, which is nothing.

The one instrument that could name what a stopped CPU is doing has still never
been fired at one: `sched::dump`'s NMI probe separates a CPU spinning with `IF`
clear from one halted with its kick undelivered from one wedged below the
interrupt layer. Take `info registers -a` over QMP before pressing Ctrl+Alt+D,
which destroys what it reports on.
What a stopped CPU is doing has still never been read off one: in a guest,
`info registers -a` over QMP (`src/qemu.rs`) gives every vCPU's `RIP`, `RFL`
and `HLT`, and is taken before injecting anything, since a keystroke revives a
halted CPU.

## The reproduction was unreachable from 2026-08-27 to 2026-09-03, and is not any more

Expand Down Expand Up @@ -167,11 +166,19 @@ calls "#156 did not fire this run, which proves nothing". What changed is that a
red now means the desktop stopped answering, which is what the declaration was
written about.

**Exit condition and owner.** Restored when #156 is fixed and a `sched::dump`
NMI probe taken on a reproduction confirms no CPU stopped taking scheduler
passes during the freeze — nothing short of that instrument distinguishes this
signature from a green run, which this entry has already shown proves nothing
either way. Owner: `kernel::sched`, the placement track that closed the
**Exit condition and owner.** Restored when #156 is fixed and a reproduction
confirms no CPU stopped taking scheduler passes during the freeze — nothing
short of that distinguishes this signature from a green run, which this entry
has already shown proves nothing either way. The instrument this exit named,
the blocked-task dump's NMI probe, is removed with its hotkey. What can read it
now is the kernel's trace diary (`kernel/src/trace.rs`), which every CPU writes
from its timer and its scheduler, printed by `/system/bin/trace`: a CPU whose
newest record stands behind the others' at the read stopped passing then, and
an `IdleEnter` as that record says it went to halt. Its rings hold 8192
records a CPU, so a CPU that kept passing has overwritten what it said before
the freeze, and the read has to be taken by a program spawned after it, which
a reproduction whose desktop stopped answering may not allow. It has never
been taken on one. Owner: `kernel::sched`, the placement track that closed the
CPU-selection half of this family (`CpuHandle::answering`,
`kernel/pure/sched/cpu.rs`) and is nearest the remaining half; held by the
orchestrator.
Expand Down
Loading
Loading