Nonstandard Errors

Albert J. Menkveld; Anna Dreber; Felix Holzmeister; Juergen Huber; Magnus Johannesson; Michael Kirchler; Sebastian Neusüß; Michael Razen; Utz Weitzel; David Abad-Díaz; Menachem Abudy; Tobias Adrian; Yacine Ait-Sahalia; Olivier Akmansoy; Jamie T. Alcock; Vitali Alexeev; Arash Aloosh; Livia Amato; Diego Amaya; James J. Angel; Jian Chen; Teodor Duevski; Petri Jylhä; Markku Kaustia; Yijie Li; Matthijs Lof; Kalle Rinne; Paul Rintamäki; Hai Tran; Wenjia Yu; Xiaoyu Zhang

doi:10.1111/jofi.13337

Nonstandard Errors

Albert J. Menkveld^*, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Sebastian Neusüß, Michael Razen, Utz Weitzel, David Abad-Díaz, Menachem Abudy, Tobias Adrian, Yacine Ait-Sahalia, Olivier Akmansoy, Jamie T. Alcock, Vitali Alexeev, Arash Aloosh, Livia Amato, Diego Amaya, James J. AngelJian Chen, Teodor Duevski, Petri Jylhä, Markku Kaustia, Yijie Li, Matthijs Lof, Kalle Rinne, Paul Rintamäki, Hai Tran, Wenjia Yu, Xiaoyu Zhang

^*Corresponding author for this work

Research output: Contribution to journal › Article › Scientific › peer-review

17 Citations (Scopus)

33 Downloads (Pure)

Abstract

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Original language	English
Pages (from-to)	2339-2390
Number of pages	52
Journal	Journal of Finance
Volume	79
Issue number	3
DOIs	https://doi.org/10.1111/jofi.13337
Publication status	Published - Jun 2024
MoE publication type	A1 Journal article-refereed

Access to Document

10.1111/jofi.13337Licence: CC BY-NC

Menkveld-et-al_The Journal of Finance_2024_Nonstandard ErrorsFinal published version, 1.33 MBLicence: CC BY

Cite this

Menkveld, A. J., Dreber, A., Holzmeister, F., Huber, J., Johannesson, M., Kirchler, M., Neusüß, S., Razen, M., Weitzel, U., Abad-Díaz, D., Abudy, M., Adrian, T., Ait-Sahalia, Y., Akmansoy, O., Alcock, J. T., Alexeev, V., Aloosh, A., Amato, L., Amaya, D., ... Zhang, X. (2024). Nonstandard Errors. Journal of Finance, 79(3), 2339-2390. https://doi.org/10.1111/jofi.13337

@article{48102a6981424ad7a8cfe8fb81e79305,

title = "Nonstandard Errors",

abstract = "In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.",

author = "Menkveld, {Albert J.} and Anna Dreber and Felix Holzmeister and Juergen Huber and Magnus Johannesson and Michael Kirchler and Sebastian Neus{\"u}{\ss} and Michael Razen and Utz Weitzel and David Abad-D{\'i}az and Menachem Abudy and Tobias Adrian and Yacine Ait-Sahalia and Olivier Akmansoy and Alcock, {Jamie T.} and Vitali Alexeev and Arash Aloosh and Livia Amato and Diego Amaya and Angel, {James J.} and Jian Chen and Teodor Duevski and Petri Jylh{\"a} and Markku Kaustia and Yijie Li and Matthijs Lof and Kalle Rinne and Paul Rintam{\"a}ki and Hai Tran and Wenjia Yu and Xiaoyu Zhang",

note = "NOA:343 Publisher Copyright: {\textcopyright} 2024 The Authors. The Journal of Finance published by Wiley Periodicals LLC on behalf of American Finance Association.",

year = "2024",

month = jun,

doi = "10.1111/jofi.13337",

language = "English",

volume = "79",

pages = "2339--2390",

journal = "Journal of Finance",

issn = "0022-1082",

publisher = "Wiley-Blackwell",

number = "3",

}

Menkveld, AJ, Dreber, A, Holzmeister, F, Huber, J, Johannesson, M, Kirchler, M, Neusüß, S, Razen, M, Weitzel, U, Abad-Díaz, D, Abudy, M, Adrian, T, Ait-Sahalia, Y, Akmansoy, O, Alcock, JT, Alexeev, V, Aloosh, A, Amato, L, Amaya, D, Angel, JJ, Chen, J, Duevski, T, Jylhä, P , Kaustia, M, Li, Y, Lof, M, Rinne, K, Rintamäki, P, Tran, H, Yu, W & Zhang, X 2024, 'Nonstandard Errors', Journal of Finance, vol. 79, no. 3, pp. 2339-2390. https://doi.org/10.1111/jofi.13337

TY - JOUR

T1 - Nonstandard Errors

AU - Menkveld, Albert J.

AU - Dreber, Anna

AU - Holzmeister, Felix

AU - Huber, Juergen

AU - Johannesson, Magnus

AU - Kirchler, Michael

AU - Neusüß, Sebastian

AU - Razen, Michael

AU - Weitzel, Utz

AU - Abad-Díaz, David

AU - Abudy, Menachem

AU - Adrian, Tobias

AU - Ait-Sahalia, Yacine

AU - Akmansoy, Olivier

AU - Alcock, Jamie T.

AU - Alexeev, Vitali

AU - Aloosh, Arash

AU - Amato, Livia

AU - Amaya, Diego

AU - Angel, James J.

AU - Chen, Jian

AU - Duevski, Teodor

AU - Jylhä, Petri

AU - Kaustia, Markku

AU - Li, Yijie

AU - Lof, Matthijs

AU - Rinne, Kalle

AU - Rintamäki, Paul

AU - Tran, Hai

AU - Yu, Wenjia

AU - Zhang, Xiaoyu

PY - 2024/6

Y1 - 2024/6

N2 - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

AB - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

UR - http://www.scopus.com/inward/record.url?scp=85190503164&partnerID=8YFLogxK

U2 - 10.1111/jofi.13337

DO - 10.1111/jofi.13337

M3 - Article

AN - SCOPUS:85190503164

SN - 0022-1082

VL - 79

SP - 2339

EP - 2390

JO - Journal of Finance

JF - Journal of Finance

IS - 3

ER -

Nonstandard Errors

Abstract

Access to Document

Other files and links

Fingerprint

Cite this