Trust · Methods
How we check things.The whole method, written down.The verification protocol.Protocol reference.
This page explains exactly how we check writing — so exactly that you could do it yourself. That's on purpose: a checker that keeps its method secret is asking you to trust it, and asking for trust is the opposite of checking. Everything below is the same method whether you pay $49 or hire us for a year: what counts as a verdict, how claims are chosen, who checks them, what the trail records, and what happens when we're wrong. It's public because verification that requires secrecy is not verification. The protocol governs every artifact this company touches — sold reports and published instruments alike. Judgment is the product; the method never was. Where a step below has a machine-checkable form, it ships as code on the downloads shelf. Normative reference for all Solonic output. Schema and staked values: /s-claims.json. Vendored enforcement: /neurath_gate.py. Verify a delivered file: mark_check in /solonic-mcp.py.
The verdict schema
Every claim that passes through the gate leaves carrying exactly one verdict. Untagged output is a bug, not a result.
- Proved
Deductively established, and machine-checked wherever the claim can be formalized.
- Verified
Evidence located, and consistent with the claim as stated.
- Open
Insufficient evidence for a verdict — stated rather than smoothed over.
- Corrected
Restated after an error; the original stays visible, struck through and dated.
- Unsupported
Evidence sought and not found. Distinct from refuted: absence of support is not presence of refutation.
On our own Ledger the schema adds Refuted — evidence against a claim we staked. Reports on your work never use it; a report tells you where support is missing, and the Ledger records where we ourselves were wrong.
What gets checked
Citations first. Every reference is resolved to a primary source and read: does the source exist, and does it say what it is cited as saying? A fabricated reference fails here; so does a real reference stretched past what it shows.
Then claims. The artifact is decomposed into its factual and inferential claims, and each is tested against sources and its own internal logic.
Load-bearing analysis. Among the claims, we identify the ones the conclusion depends on — the claims that, if they fell, would take the argument with them. These are named explicitly in every report, because a verdict matters in proportion to the weight the claim carries.
At the deepest tier, attack. Adversarial review tries to break the argument outright, and every claim that survives is delivered with a named falsifier: the specific finding that would overturn it. A claim without a falsifier hasn't been attacked; it's been admired.
Who checks: the panel
The checking is done by a panel of independent AI model families, run under human oversight. Independence is the entire design: correlated checkers don't add up — ten copies of the same reviewer are one reviewer with a louder voice. So the panel is constructed for decorrelation: families with different training lineages and different failure modes, so that an error must fool reviewers that fail differently to get through.
Two disciplines follow. Generation and verification are separated: the family that drafts a thing never sits in judgment of it. And every report names the families that actually sat — not a fixed roster claimed in the abstract, but the roster of record for that artifact.
We also say the unflattering part out loud: correlation eats panels. Nine families at ρ ≈ 0.2 behave like about three independent ones — we design for that, and we publish the correlation we find rather than the headcount we'd prefer.
Machines are not the only anchor. A standing panel of human researchers — the Heliaia — adjudicates ground truth, so the verifier's agreement rate with experts becomes a measured number rather than a promise, and attacks our own artifacts before they ship. A chain that only checks itself converges on confidence, not correctness.
The audit trail
A verdict you can't re-run is an opinion with a stamp on it. So every claim in a report carries its trail: what was checked, against what, and what the verdict was — the sources consulted, the tests applied, and the reasoning that connects them. A third party with the report can re-run the judgment, not just read the conclusion.
The same discipline is vendored as code: neurath_gate.py refuses to emit any claim without a tag, evidence, and a falsifier — it fails closed. It is the gate this company runs, published so you can run it on us.
Scoring, for humans and machines alike
Judgments that can resolve, do: when a forecastable claim settles, the judgments made about it — by model families and by human panelists — are scored against the outcome by proper scoring rules, under which the profitable strategy is reporting what you actually believe. Calibration records accumulate from resolution, not reputation. Any measure used for decisions invites gaming — Campbell's law sits in our own Atlas tagged as directly self-applicable — and proper scoring plus recruited disagreement is the oldest defense there is.
Measuring our own accuracy
We have not yet published our own error rate — and we say so, because a verification service whose accuracy is unmeasured is making a promise, not a claim. We are running Jürge against a validation set of papers with known peer-review outcomes, to measure false-positive rate, false-negative rate, and correlation with expert judgment; the results and the methodology will be published here when complete. Until then, our track record is the Ledger and the errata — both public, both dated.
Classes — if you'd rather be taught
We teach the whole loop — generation, gates, ledger discipline, errata practice — to teams who want to run it themselves. Cohorts are small and scheduled by demand. Write to us and say what you work on.
Self-application, and what happens when we're wrong
The gate runs on us. Every instrument on the research shelf passed the same review we sell, and our flagship went through the deepest tier with the findings published — the excerpt is on the Verify page, unflattering verdicts included.
Corrections are structural, not shameful. A corrected claim is restated with the original left visible, struck through and dated. Corrections to our published claims are entered in the Ledger the day the evidence moves; corrections to this site itself are logged in the errata. Overturn a staked claim and the entry carries your name — the standing challenges exist to make that easy.
The method's own status line: this page describes the protocol as practiced; where a number is staked (counts, rates, turnarounds), the single source is s-claims.json, and this page defers to it.