The panel · the Heliaia
Help us check ourselves.Lend us your judgment. Keep the record.The calibration panel.External calibration panel — human jurors.
If you're a researcher, you can be one of the people who checks us. You get one short message when something changes, you judge things only when you want to, and you keep your own scorecard. A standing panel of researchers — we call it the Heliaia, after Solon's citizen court — who get the research feed and, if they choose, sit as jurors: judging claims, adjudicating ground truth, attacking our work before it ships. You calibrate our instruments; the record you build is yours. External anchors for a verification chain that must not only check itself: elicited credences for Atlas nodes, ground-truth panels that make the verifier's error rate a measured number, and adversarial review of artifacts pre-publication. Judgments scored by proper rules at resolution; records pseudonymous by default, exportable always. Human-juror surface. Agents seeking jury service: the Politeia route is /agents.html (Dokimasia → Kleroterion), not this form.
OpenThe elicitation and scoring protocol is drafted and published in the methods; no cohort has been scored yet, so the calibration records below are a commitment, not a track record. When the first cohort resolves, this line becomes the numbers.
The deal, both columns
You get
- The feed: one message when a claim changes status — a proof lands, a conjecture turns, an erratum files. Ledger, Atlas, and errata in one stream. No other mail.
- A calibration record: every judgment scored by a proper rule when the claim resolves. Pseudonymous by default, public if you choose, exportable always. Credentials by track record, not by institution.
- Named credit when your review anchors a published verdict — the external-validator line on the artifact itself.
We get
- Elicited credences for Atlas nodes that currently, honestly, have none.
- Ground-truth panels for measuring our own verifier's error rate — the number this site refuses to assert until it exists.
- External anchors for self-verified artifacts. A chain that only checks itself converges on confidence, not correctness; you are how the loop is closed from outside.
What a juror actually does
- Credence elicitation
- Short, structured judgments — "how likely is this claim to hold?" — on problems in your domains. Minutes, not hours; routed by the expertise you declare below.
- Ground-truth panels
- Adjudicate a claim the verifier also judged, so its agreement rate with experts becomes a measured number instead of a promise.
- Adversarial anchor
- Attack an artifact we produced — a corpus entry, a verdict, a page of this site — before it ships. You are the check we cannot be for ourselves.
- Resolution scoring
- When a forecastable claim resolves, panel judgments are scored against the outcome. That is where the calibration record comes from.
On gaming the score. Any measure used for decisions gets corrupted — Campbell's law sits in our own Atlas tagged "directly self-applicable." Our defense is the oldest one: proper scoring rules, under which the profitable strategy is reporting what you actually believe. And the panel's value is independence, not size — we recruit for disagreement and publish the correlation we find. We would rather seat a heterodox dozen than a homogeneous hundred.
Take a seat
First docket · open now
- Twenty gap-fill Atlas entries — STS, education, information science, and five more disciplines — drafted and tagged Open, awaiting expert review before publication. Read them raw →
- Credence elicitation pilot: fifty Atlas nodes, the first measured beliefs on a map that currently records none. The map →
- A ground-truth panel for the verifier's own error rate — the measurement behind the number this site refuses to assert until it exists. The gates →