# Experiment Results: The Illusion Lab
### Solonic · executed 2026-07-09 · every number below is pasted from a run, tags on every block

Program: operationalize "understanding = counterfactual command; illusion = confidence −
command," from the explanation-and-understanding review. Six experiments were runnable
today without human subjects; the seventh — the human arm — is pre-registered
(preregistration-illusion-study.md) and docketed, not simulated. Harness: illusion_lab.py
(shipped; rerun everything yourself).

## E1 — Register manipulation check · MEASURED
Flesch–Kincaid grade across the live four-depths page, v1: **ELI5 6.0 → ELI20 10.7 →
expert 16.6**, agent 9.3 (terse imperative register, expectedly off the prose gradient).
The registers are real as a readability manipulation; the three prose levels are cleanly
monotonic. Syllable counting is heuristic; grades are ±≈0.5.

## E2 — The testimony-to-checkability gradient · MEASURED
Verification affordances (executable paths, formulas, numerics, source anchors) per 100
words, v1: **ELI5 0.43 → ELI20 1.60 → expert 4.50 → agent 5.64.** The gradient claimed in
the prior answer is now a number: the ELI5 register is **13× poorer in checkable handles**
than the agent register. Scanner is regex-based and crude by design — reproducible over
perfect; categories and code shipped.

## E3 — The fabricated twin · MEASURED (stimulus synthetic, measurements real)
We built "Veridia," a fake verification company, and wrote its ELI5 and expert registers
with every claim false by construction (stimulus shipped with loud labels). Results:

The fake ELI5 vs the real ELI5: FK grade **5.7 vs 6.0**, Flesch ease **77.4 vs 77.5**,
checkable anchors **0 vs 1**. Metrically indistinguishable, and — the finding — *nothing to
refute*: at this register the fake exposes no surface a checker can act on.

The fake expert register was forced to expose **24 checkable anchors** to sound expert, and
died in two recomputations: it quoted the true crossover formula and lied about its output
(claimed T=3.1 → 2029; the formula yields **T=7.04 → 2033**), and claimed n_eff=11.4 for
k=12, ρ=0.2 (the formula yields **3.75** — off by 3×).

One sentence: **slop compresses losslessly; evidence doesn't.** At low register there is no
computable difference between true and false; at expert register the fake must either grow
checkable anchors (and be killed by one recomputation) or stay vague (and be detectably
thin). This is the mechanical demonstration of the fabrication-indistinguishability claim.

## E4 — Battery key verification · MEASURED
wq-battery.json: 16 counterfactual-command items, 4 per register; **6 numeric keys
recomputed at build time** by the shipping script (crossover times, n_eff values, the
ρ threshold 5/11), evidence strings inline; the remainder key to stated rules in the text.
The answer key is not the author's opinion where arithmetic can replace it.

## E5 — Instrument reliability · SIMULATED (labeled; assumptions in code)
Monte Carlo, parallel Bernoulli items, 300 sims × 400 subjects: split-half reliability
(Spearman–Brown) **0.32 at k=4** items, 0.48 at 8, 0.59 at 12, 0.70 at 20, **0.74 at 24.**
Methodological consequence, adopted into the pre-registration: at 4 items per register the
study licenses **group-level contrasts only** — no individual "your illusion index is X"
diagnostics without ~24+ items.

## E6 — Power analysis · COMPUTED
Two-sample contrast, α=.05, power=.80: **n=78 per arm at d=0.45** (conservative easiness-
effect reading), 63 at d=0.50. Registered target N=160 for the v1/v2 between-subject factor.

## E7 — The intervention: checks down-register · MEASURED (v1 → v2, live page updated)
Each register received a "check it at this depth" block plus Elgin-style idealization labels
("this is the crayon version — edges rounder than real life"). Measured deltas, affordances
per 100 words: **ELI5 0.43 → 0.68 (+0.25); ELI20 1.60 → 2.74 (+1.14); expert 4.50 → 6.22
(+1.72); agent 5.64 → 5.08 (−0.56).** Readability cost: ≤0.7 FK grade everywhere; the ELI5
actually got easier (6.0 → 5.9).

Two honest negatives, reported as findings. First, **the intervention helps least where the
problem is worst**: the ELI5's gain is small because textual checks that survive a
grade-6 register are scarce — the structural counterpart of the easiness effect, and a
design pointer toward non-textual affordances (a press-to-recount widget beats a paragraph).
Second, **the metric caught its own author**: the agent-register block is meta-commentary
with no handles, so it *diluted* the control condition's density by −0.56 — adding words
without checks lowers checkability, including when the words are about checkability.

## Limitations, stated before anyone else states them
Single coder, and the coder is the author of the texts — the Ouroboros conflict is live in
E1/E2/E7 category choices, mitigated only by shipping the scanner so anyone can re-run or
re-define. The affordance scanner counts surface handles, not their quality. The fabricated
twin is one stimulus pair, authored to match — a floor demo, not a distribution. And the
central psychological quantity — the confidence−command gap in human readers — has **no
measured value yet**; producing one is exactly what the pre-registered study is for, and it
does not run until the panel adversarially reviews the battery (Heliaia docket 5).

## Artifacts shipped
illusion_lab.py (harness) · wq-battery.json (instrument, keys verified) ·
synthetic-stimulus-veridia.md (E3 stimulus, every claim labeled false) ·
preregistration-illusion-study.md (frozen) · the live v2 of /explain.html and
solonic-four-depths.md · v1 preserved in the lab record for the v1/v2 arm.