11%Cancer biology replication rate (Reproducibility Project)
36%Psychology replication rate (OSC 2015)
61%Economics replication rate (Camerer et al. 2016)
p<0.05The threshold that broke science
When scientists do an experiment and get a result, other scientists are supposed to be able to do the same experiment and get the same result. But when researchers went back and tried to repeat famous psychology and cancer studies, most of them didn't work — the results just didn't happen again. For cancer research, only 11 out of every 100 studies could be repeated. This is a huge problem because other research (and drugs) get built on these unreplicable foundations.
The replication crisis refers to the finding that many published scientific results cannot be independently reproduced. The landmark Open Science Collaboration (2015, Science) replicated 100 psychology studies: only 36% achieved comparable results. Camerer et al. (2016) replicated 18 economics studies: 61% replicated. The Reproducibility Project: Cancer Biology (eLife, 2021) attempted to replicate 50 cancer studies: only 11% fully replicated with similar effect sizes. Key driver: publication bias (journals prefer positive results), p-hacking (testing many hypotheses until p<0.05), and small sample sizes that produce unreliable estimates.
OSC 2015 (Science, DOI: 10.1126/science.aac4716): 100 psychology replications. Replication rate by original p-value: originally p<0.001: 50% replicate; originally p=0.04-0.05: 22% replicate. Effect size comparison: median replicated effect 50% of original. RP:CB (eLife 2021, DOI: 10.7554/eLife.71601): 50 cancer biology studies attempted, 23 informative, only 2-4 (8-17%) fully replicated, 11% mid-estimate. Heckman (2022): FDA clinical trial replication rate ~50% for Phase 3 (effectively filtered by registration, blinding, power requirements). p-hacking: with 20 independent hypothesis tests, probability of p<0.05 by chance = 64%. Pre-registration as solution: dramatically increases replication rates when enforced (Registered Reports format: 85% replication rate per meta-analysis, Scheel et al. 2021).
OSC 2015: science.org/doi/10.1126/science.aac4716. Camerer et al. economics: science.org/doi/10.1126/science.aaf0918. RP:CB (cancer biology): elifesciences.org/articles/71601. Scheel et al. Registered Reports: royalsocietypublishing.org/doi/10.1098/rsos.211598. Pre-registration platform: osf.io. metafor R package for meta-analysis: metafor-project.org. AllTrials campaign (mandatory trial reporting): alltrials.net. The replication crisis has a Wikipedia article with comprehensive bibliography: en.wikipedia.org/wiki/Replication_crisis.