Every check here proves it can still fail on each push
Every check in this repo makes two claims: it passes on a good tree, and it fails on the defect it guards. CI tested the first claim on every push. The second was tested once, by hand, in the PR that added the check, and 20 commit bodies record a "Controls:" run that never ran again. In between, three checks read zero files for up to 60 days while every build went green.
A green check is half a measurement
A check that passes says one of two things: the tree is clean, or the check stopped looking. From the outside they read the same. Invariant 8c fails the build when a Garage page has no entry in site-manifest.json. The www/ split in #457 (2026-08-18) left its walk pointed at public/garage, which still held av2/ and lean/ assets but no .html. The directory existed, so the walk succeeded, found nothing, and reported nothing unregistered. It inspected 0 of 39 pages for 60 days.
The only way to tell the two apart is to plant the defect and see whether the check notices. That's a control: the edit that reintroduces the bug, the check that must catch it, and the words the check prints when it does. Since 2026-10-30, tools/lib/controls.ts keeps each one as data, and bun run controls plants every one on every push.
The bench
Four checks from this repo, rebuilt small enough to run in your browser. Each runs twice, the way the harness runs it: once on a good tree, which has to pass, and once with its defect planted. The first box replays the change that blinded the real check. The second applies the fix it got.
Press the first preset. Every good tree still passes, which is all validate could see that week, and every planted defect sails through. Then turn on the walk's drift and its floor together: the good tree goes red. The floor leaves the walk reading the wrong directory and refuses to call the empty walk a pass, so the move that blinds the check fails the build it lands in.
Four verdicts, and why the words matter
The harness reads each planted run as one of four verdicts. CAUGHT: the check went red and printed its own line. BLIND: it passed with the defect in place, which is the finding the harness exists for. WRONG REASON: it went red, but something else failed first. STALE: the text a mutation anchors on has moved, so the defect can't be planted. A contract test in validate fails on STALE, so the PR that refactors a guarded file has to move its control too.
WRONG REASON is why every control quotes its check. The first serendipity control planted its conflict marker as bare code. Bun's parser failed the build before the scan ran, so on the tree before #1390 the harness read WRONG REASON where the truth was BLIND. The marker now sits inside a template literal, where the module still parses and only the scan can see it. Tick "plant it as bare code" on the bench to watch the same thing happen.
Before any control runs, each distinct check runs once on an unplanted clone and has to pass. A check that's already red proves nothing by going red again. The first three were found by accident. Replayed against the trees before #1390 and #1408, the harness reads BLIND on all three, so it would have caught each one the day it opened. The last three it found itself, within a day of landing. Gotcha 13's walker is the expensive one: it guards the Each control plants into its own copy-on-write clone of the worktree: one The registry went from 11 controls on 2026-10-30 to 25 the next day and 44 on 2026-11-09, covering 11 gotchas. On 2026-11-09 all 44 read CAUGHT across 28 distinct checks in 94.1 s on a 12-core M3 running four clones at a time. CI ran the 42 then on main in 81.3 s. The job reports beside A control proves a check catches the one defect someone wrote down. A defect nobody planted stays invisible. The deploy-pin gap had no check at all, and the harness found it only because writing that control sent someone looking. Coverage is still a list a person keeps, so gotcha 90 asks every new check to arrive with its control in the same PR, quoting the line the check prints when it catches.What the first runs found
check what changed under it what it read blind for found invariant 8c: every page registered #457 moved the pages to src/pages (2026-08-18)0 of 39 pages 60 days by luck, #1390 conflict-marker scan #523 made serendipity TypeScript (2026-08-23) 0 of 3 files 55 days by luck, #1390 taskbar-floor agreement #1369 spelled the floor through --ad (2026-10-16)0 of 3 copies 1 day by luck, #1408 gotcha 13's Response walker #494 typed two inits as ResponseInit (2026-08-20)2 inits misread as Responses 70 days writing a control, #1488 deploy-command pins no check read the 4 deploy scripts outside infra.json0 of 4 scripts always writing a control, #1488 22 git scans each listed files from the index, which misses a file before git add19 of 22 blind on an unadded file since each was written planting one per scan, #1490 encodeBody flag that a rebuilt Response drops, and for 70 days a rebuilt Response on the noindex path could drop that flag and the walker would pass it.What it costs
clonefile(2) call, about 130 ms on 2026-10-29. Other sessions work in the same tree, so a planted conflict marker must never land where one of them could commit it. In CI the checkout is private, so the runner plants in place and puts every touched file back.validate with continue-on-error. A blind check is a finding about the checks, so it never holds up a release.What it can't see
