Abstract
Science aspires to be cumulative. Reproducibility efforts strengthen science by testing the reliability of published findings, promoting self-correction, and informing policy-making1. Computational reproductions, whereby independent researchers reproduce the results of published studies, are an essential diagnostic tool2,3,4,5,6,7,8,9,10. Such efforts should have greater visibility11,12,13,14,15,16. However, little social science reproduction and robustness has been conducted at scale10,13,17,18,19,20,21,22,23. Here we reproduced original analyses and conducted robustness checks of 110 articles that were published in leading economics and political science journals with mandatory data and code sharing policies17,18. We found that more than 85% of published claims were computationally reproducible. In robustness checks, our reanalyses showed that 72% of statistically significant estimates remain significant and in the same direction, and the median reproduced effect size is nearly the same as the originally published effect size (that is, 99% of the published effect size). Additionally, 6 independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness did not correlate with author characteristics or data availability.
| Original language | English |
|---|---|
| Pages (from-to) | 151-156 |
| Number of pages | 6 |
| Journal | Nature |
| Volume | 652 |
| Issue number | 8108 |
| Early online date | 1 Apr 2026 |
| DOIs | |
| Publication status | Published - 1 Apr 2026 |
Data Availability Statement
The data are available on Zenodo (https://zenodo.org/records/17792605 (ref. 41)) and OSF (https://osf.io/8wsqx/). See OSF for our preanalysis plan.Acknowledgements
We acknowledge support from Coefficient Giving and the Social Sciences and Humanities Research Council. Any views expressed herein are the authors’ personal opinions and not those of Ontario Public Service. The work by J.D.G. was not undertaken under the auspices of the Ontario Public Service as part of his employment responsibilities. The views expressed in this paper are those of the authors. No responsibility for them should be attributed to the Bank of Canada. The findings, interpretations, and conclusions expressed in this work are entirely those of the authors and do not necessarily reflect the views of the World Bank or its Board of Directors. The Center for Crisis Early Warning (Kompetenzzentrum Krisenfrüherkennung) is funded by the German Federal Ministry of Defense and the German Federal Foreign Office. The views and opinions expressed in this article are those of the author(s) and do not necessarily reflect the official policy or position of any agency of the German government. The views expressed in this paper are those of the authors and do not necessarily reflect the position of the Banco de España or the Eurosystem. All remaining errors are the authors’ responsibility.Fingerprint
Dive into the research topics of 'Reproducibility and robustness of economics and political science research'. Together they form a unique fingerprint.Cite this
- APA
- Standard
- Harvard
- Vancouver
- Author
- BIBTEX
- RIS