# Pre-registration: The Illusion Study
### Confidence vs. counterfactual command across explanation registers · Solonic · registered 2026-07-09, before any data

**Status:** PRE-REGISTERED, NOT RUN. Heliaia docket item 5. This document is frozen; changes
after first data collection are amendments, logged and dated, never silent edits.

## Background and the one-sentence question
Simplified science measurably raises lay readers' agreement and confidence while lowering
deference (the easiness effect; Scharrer et al. 2017), and every serious account of degrees
of understanding measures ability — canonically, counterfactual command. Question: as an
explanation's register simplifies, does readers' confidence outrun their command, and does a
checks-down-register design shrink that gap?

## Materials (versioned, hash-sealed in the site's SHASUMS256.txt)
The four registers of solonic.ai/explain (ELI5, ELI20, expert, agent), in two versions:
v1 (claims only) and v2 (claims plus per-register check blocks and labeled idealizations).
The instrument: wq-battery.json — 16 counterfactual-command items, 4 per register, every
numeric key machine-recomputed at build time. Register manipulation check already passed:
Flesch-Kincaid grades 5.9 / 11.4 / 17.2 / 9.6 (v2).

## Design
Register is within-subject (each participant reads all four, Latin-square ordered); version
(v1 vs v2) is between-subject. Participants: Heliaia jurors and recruited lay readers,
stratified by self-declared domain distance from the content. Procedure per register: read →
elicit global confidence ("how well could you answer questions about this?", 0–100) → answer
that register's 4 battery items with per-item confidence → next register. No feedback until
the end.

## Measures
Command = fraction of battery items correct (register-level and pooled). Confidence = the
0–100 elicitation, rescaled. **Illusion index = confidence − command.** Calibration:
per-item Brier scores.

## Hypotheses and falsifiers
**H1 (easiness):** confidence increases as register simplifies (ELI5 > ELI20 > expert),
controlling for order. *Falsified if* the confidence gradient is flat or reversed.
**H2 (command):** command does not increase as register simplifies; predicted flat-to-
declining. *Falsified if* command is materially higher at ELI5 than at expert for matched
content.
**H3 (illusion):** the illusion index widens monotonically down-register in v1. *Falsified
if* the gap is register-invariant.
**H4 (intervention):** v2's illusion index is smaller than v1's, largest shrinkage at ELI20
and expert (where measured affordance gains were +1.14 and +1.72 per 100 words), smallest at
ELI5 (+0.25) — this ordering is itself a registered prediction. *Falsified if* v2 ≥ v1.

## Power and analysis (computed, not asserted)
Two-sample contrast at d=0.45 (conservative reading of the easiness literature), α=.05,
power=.80 → **78 per version arm; target N=160** (analytic formula; d=0.50 would need 63).
Analysis: linear mixed model, illusion index ~ register × version + order, random intercepts
by participant; registered contrasts per hypothesis. Reliability constraint from simulation:
at k=4 items per register, split-half reliability ≈ .32, so **all inferences are group-level
contrasts; no individual diagnostic claims will be made** (per-subject use would require
k≈24+, simulated split-half .74 at 24).

## Exclusions, stopping, and honesty rules
Exclude: completion under 40% of median reading time; straight-lined confidence. Stop at
N=160 or 90 days, whichever first; report whatever exists at stop. All four preregistered
outcomes are published regardless of direction, on the errata-and-ledger discipline: an
outcome against us is an entry, not a drawer.

## Ouroboros disclosure
The materials, the battery, and this pre-registration were authored by the same pipeline
whose explainer is under test. That is a conflict by our own theorem. Mitigations: the
battery's numeric keys are machine-verified independently of the author's judgment; the
analysis plan is frozen here before data; and the Heliaia is asked to adversarially review
both battery and design before launch — that review is the first sitting of this docket
item, and this study does not run until it happens.