chore(external-validity): retire design-era working material from the study tree

[willhea]

Repository hygiene for the PDF external-validity study. No methodology changes. The study is not being declared complete: Run 2 is still the intended next step, and everything its execution path needs is retained.

Policy

Research artifacts are working material, not automatically permanent repository material. At closure, retain only artifacts needed to reproduce a consequential result, enforce an invariant, document a durable decision, or serve as a frozen input. Git history preserves the investigative record.

Applied strictly — "useful history" and "was once reviewed" are not retention reasons — but decided by tracing consumers, never by filename or age. The rule is now standing, in AGENTS.md under a new "Research artifacts" section, so future research work does not re-accumulate by default.

Removed

38 files, 797 KB. Nothing removed is read by any code, gate, test or CI job; that was verified mechanically, not by eye (every basename grepped repo-wide, with the producing probe's own output-path line excluded as a write target rather than a read).

category	n	why it fails the retention test
generated probe evidence (results/xNN_*.json)	20	recorded only that an earlier run passed; no consumer, and each is regenerated by a probe still in the tree
development raster images (results/x19_raster_dev/)	16	referenced only by x19's own evidence file
obsolete probes	2	see below

probes/x06_m6_feasibility.py measured whether M6 could be adjudicated inside a 6–10 line region. A20 STRUCK M6 and HARNESS-PLAN implements "§6 (M0–M9 minus M6)", so the question is no longer in the methodology; it also drives the superseded pdfium_hybrid/reconstruct_hybrid arms rather than the frozen H/X pair, so it exercises no current component. A17.1 transcribes its table inline.

probes/x19_raster_edge_diagnostic.py asked whether the visible left edge is displaced from the geometric x0. It states no pass threshold by construction, and the ruling it informed (A30.3 stands, no tolerance) is frozen in A32.

Regenerability was measured, not assumed: every committed probe artifact in the study was re-run before deletion and rewrote byte-identically, leaving the tree clean.

Kept

The audit reversed several of the obvious-looking deletion candidates. The ones worth naming:

• results/design_runs/*.log — these look like run logs and are not. x05 derives the design-exposure list from them, and that list disqualifies documents from the frozen population (F3) and feeds G4. A primary source.
• results/x26_control_oracle.json — looks like historical probe output; G6 defects ORACLE_INTEGRATION_NOT_VERIFIED without it. A committed expected-output oracle.
• results/x00_design_pilot.json, results/x02_oracle_reference_defects.json — cited as MEASURED by PRE-REGISTRATION.md, which is byte-frozen (F4 depends on its last-modifying commit) and can therefore never be given a pointer to git history.
• results/x09_…, x11_…, x13_… — named by retained METHODOLOGY_SURFACE files. Repointing those would mean editing a result-bearing file for a cosmetic reason.
• Every other xNN probe — each is the executable control over a retained live component.

The line that decided the ambiguous cases: a dangling reference is acceptable only where the referring document can be appended to. The ledger can record where a removed file resolves; a byte-frozen pre-registration cannot, so what it cites stays.

Governance (F9)

Deleting a .py/.md under the study tree is visible to F9 whether or not the file is in METHODOLOGY_SURFACE. Handled through the existing mechanism, not around it:

• A46 declares deletion commit 4b94672, naming all three protected files it touched.
• Filed SUBSTANTIVE, affects_scoring_rule: false. Not TOOLING: all three files are already SUBSTANTIVE-declared and x04 refuses a file declared under both. Not CLERICAL: that would misdescribe removing two executable probes as a typo fix.
• No historical ledger prose is rewritten. A17/A19/A21/A22/A23/A24.2 and A41 keep their MEASURED links; A46 records that they resolve at 4b94672~1 and that each amendment already transcribes its numbers inline.
• F9 was confirmed to actually fire on the deletion commit before A46 landed, so the declaration is closing a real red gate rather than a hypothetical one.

Nothing changed in membership, scoring rules, thresholds, architecture rules or frozen study semantics. PRE-REGISTRATION.md is untouched.

Safety

Baseline (origin/develop = d7c169d, PR #637/A45 merged) vs this branch — identical except F9's expected amendment count:

gate	baseline	after
x04 freeze/readiness	F1–F11 COMPLETE · G1–G6 PASS · BOUNDARY ABSENT	identical (F9 45→46 amendments, 82→83 commits)
x04 --self-test	36/36	36/36
x27 score_metrics	194/194	194/194
x29 execute_study	96/96	96/96
x23 control fixtures	55/55	55/55
x22 score input contract	52/52	52/52
x2_verify	X2-a PASS, X2-b PASS / 1559 boundaries	identical
full pytest	—	3423 passed, 33 skipped, 15 xfailed
ruff check . / ruff format --check	—	clean / 241 formatted

Every other retained invariant owner was also run, since a removed artifact that turned out to be consumed would surface here: x21 117/117 · x28 92/92 · x15 64/64 · x17 56/56 · x26 12/12 · x24 16/16.

x26 and x2_verify write artifacts that are retained and consumed — both rewrote byte-identically, so G6's ORACLE_EVIDENCE_STALE digest binding and G2's blob binding stay valid. The .gitignore guard was then confirmed live: after nine probe runs the working tree is clean, the regenerated transient artifacts show as ignored, and every negated artifact stayed tracked.

The 33 pytest skips are benign fixture-shape skips (shell bills with no amounts) and update-mode guards; none is a corpus-degraded or silently-skipped schema gate. This worktree has no bills_corpus/bills_bulk_text symlinks, which affects only provision-matching research tests unrelated to this change.

Explicitly verified: frozen population unchanged (17 members, membership blob d2731d6a == frozen blob); holdout bytes unchanged (F2/F11); G5 surface still 15 files, none missing; G6 still N-A 8 / N-B 8 / N-C 4, 0 defects; the A45 cross-engine authority repair still live (x29 96/96 includes its controls); execution boundary still ABSENT, no EXECUTION-START.json created, no confirmatory artifact written, no holdout extraction run, and no 1.1 GB artifact introduced.

Falsification

Readiness must not go green merely because a control was deleted, so that was tested rather than argued. Each fault was injected and reverted, tree clean after each:

• A — over-deletion of a frozen input. Removing results/x26_control_oracle.json → G6 FAIL ORACLE_INTEGRATION_NOT_VERIFIED, readiness CLOSED. Removing results/design_runs/ → x05 FATAL rc=2, the design-exposure list becomes unreproducible. Both keeps are load-bearing, and over-deletion of this class is detectable.
• B — missing live invariant owner. Removing probes/x2_verify.py → G2 FAIL (blob binding and live verifier exit 2), G5 FAIL MISSING 1/15, F9 red, readiness CLOSED. Three independent gates catch it.
• C — deleted historical artifact. Removing results/x13_x_arm.json → 17/17 PASS, FREEZE COMPLETE, READINESS OPEN, BOUNDARY ABSENT. Semantically unchanged.

A and C together are the point: the gates fail on a frozen input and are indifferent to a historical artifact, so the two classes are distinguishable rather than uniformly invisible. Because C is invisible to the gates by design, the no-consumer proof for that class is the mechanical reference sweep above, not the green suite.

Also

.gitignore gains the study's transient probe evidence. Running any xNN probe rewrites its own evidence file, so after this cleanup those reappear as untracked and a later git add -A would silently restore the retired material, looking like ordinary probe output rather than a reverted decision. The seven retained artifacts are negated, and both directions were checked with git check-ignore rather than assumed.

No new files are added anywhere; the temporary audit inventory was kept outside the repository and is not committed.