How can we make sound replication decisions?

Clintin P. Davis-Stober; Alexandra Sarafoglou; Balazs Aczel; Suyog H. Chandramouli; Timothy M. Errington; Sarahanne M. Field; Ayelet Fishbach; Juliana Freire; John P.A. Ioannidis; Klaus Oberauer; Franco Pestilli; Susanne Ressl; Daniel J. Schad; Judith ter Schure; Katya Tentori; Don van Ravenzwaaij; Joachim Vandekerckhove; Odd Erik Gundersen

doi:10.1073/pnas.2401236121

How can we make sound replication decisions?

Clintin P. Davis-Stober^*
, Alexandra Sarafoglou
, Balazs Aczel
, Suyog H. Chandramouli
, Timothy M. Errington
, Sarahanne M. Field
, Ayelet Fishbach
, Juliana Freire
, John P.A. Ioannidis
, Klaus Oberauer
, Franco Pestilli
, Susanne Ressl
, Daniel J. Schad
, Judith ter Schure
, Katya Tentori
, Don van Ravenzwaaij
, Joachim Vandekerckhove
, Odd Erik Gundersen

^*Corresponding author for this work

Department of Information and Communications Engineering

Research output: Contribution to journal › Article › Scientific › peer-review

3 Citations (Scopus)

19 Downloads (Pure)

Abstract

Replication and the reported crises impacting many fields of research have become a focal point for the sciences. This has led to reforms in publishing, methodological design and reporting, and increased numbers of experimental replications coordinated across many laboratories. While replication is rightly considered an indispensable tool of science, financial resources and researchers' time are quite limited. In this perspective, we examine different values and attitudes that scientists can consider when deciding whether to replicate a finding and how. We offer a conceptual framework for assessing the usefulness of various replication tools, such as preregistration.

Original language	English
Article number	e2401236121
Number of pages	10
Journal	Proceedings of the National Academy of Sciences of the United States of America
Volume	122
Issue number	5
DOIs	https://doi.org/10.1073/pnas.2401236121
Publication status	Published - 2025
MoE publication type	A1 Journal article-refereed

Funding

This work was supported by a Amsterdam Brain and Cognition (ABC) project grant to A.S. (ABC PG 22); C.P.D.-S. was supported by Department of Defense grant #W81XWH2110173 and NIH award #R01AA030914; J.V. was supported by the NSF grants #1230118, #1658303, #1850849, and #2051186; A.F. was supported by The University of Chicago, Booth School of Business; S.R. was supported by The University of Texas at Austin, College of Natural Sciences; D.v.R. was supported by the Dutch scientific organization Vidi fellowship grant (016.Vidi.188.001); F.P. was supported by U.S. NSF awards (1916518, 1912270, 1636893, and 1734853), U.S. NIH awards (NIH; R01MH126699, R01EB030896, and R01EB029272), the Wellcome Trust award (226486/Z/22/Z), a Microsoft Investigator Fellowship, and a gift from the Kavli Foundation; J.F. was supported by the NSF grants#2106888 and #2146306 and Defense Advanced Research Projects Agency D3M and Assignment Satisfaction Key - Enlisted Module programs; O.E.G. was supported by the Norwegian Research Council through the grant agreement 329730; S.H.C. was supported by the Finnish Center for Artificial Intelligence, and Academy of Finland (328813) and acknowledges the support from the Jorma Ollila Mobility Grant by Nokia Foundation. The views, opinions and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

Keywords

methodology
reform
replication
reproducibility

Access to Document

10.1073/pnas.2401236121Licence: CC BY

davis-stober-et-al-2025-how-can-we-make-sound-replication-decisionsFinal published version, 1.37 MBLicence: CC BY

Cite this

Davis-Stober, C. P., Sarafoglou, A., Aczel, B., Chandramouli, S. H., Errington, T. M., Field, S. M., Fishbach, A., Freire, J., Ioannidis, J. P. A., Oberauer, K., Pestilli, F., Ressl, S., Schad, D. J., ter Schure, J., Tentori, K., van Ravenzwaaij, D., Vandekerckhove, J., & Gundersen, O. E. (2025). How can we make sound replication decisions? Proceedings of the National Academy of Sciences of the United States of America, 122(5), Article e2401236121. https://doi.org/10.1073/pnas.2401236121

@article{53c090d91d224d2e84ee0ce5aca9a0c5,

title = "How can we make sound replication decisions?",

abstract = "Replication and the reported crises impacting many fields of research have become a focal point for the sciences. This has led to reforms in publishing, methodological design and reporting, and increased numbers of experimental replications coordinated across many laboratories. While replication is rightly considered an indispensable tool of science, financial resources and researchers' time are quite limited. In this perspective, we examine different values and attitudes that scientists can consider when deciding whether to replicate a finding and how. We offer a conceptual framework for assessing the usefulness of various replication tools, such as preregistration.",

keywords = "methodology, reform, replication, reproducibility",

author = "Davis-Stober, \{Clintin P.\} and Alexandra Sarafoglou and Balazs Aczel and Chandramouli, \{Suyog H.\} and Errington, \{Timothy M.\} and Field, \{Sarahanne M.\} and Ayelet Fishbach and Juliana Freire and Ioannidis, \{John P.A.\} and Klaus Oberauer and Franco Pestilli and Susanne Ressl and Schad, \{Daniel J.\} and \{ter Schure\}, Judith and Katya Tentori and \{van Ravenzwaaij\}, Don and Joachim Vandekerckhove and Gundersen, \{Odd Erik\}",

note = "Publisher Copyright: Copyright {\textcopyright} 2025 the Author(s)",

year = "2025",

doi = "10.1073/pnas.2401236121",

language = "English",

volume = "122",

journal = "Proceedings of the National Academy of Sciences of the United States of America",

issn = "0027-8424",

publisher = "National Academy of Sciences",