A Plea for Neutral Comparison Studies in Computational Sciences

Anne-Laure Boulesteix, Sabine Lauer, Manuel J A Eugster

    Research output: Contribution to journalArticleScientificpeer-review

    33 Citations (Scopus)
    172 Downloads (Pure)

    Abstract

    In computational science literature including, e.g., bioinformatics, computational statistics or machine learning, most published articles are devoted to the development of "new methods", while comparison studies are generally appreciated by readers but surprisingly given poor consideration by many journals. This paper stresses the importance of neutral comparison studies for the objective evaluation of existing methods and the establishment of standards by drawing parallels with clinical research. The goal of the paper is twofold. Firstly, we present a survey of recent computational papers on supervised classification published in seven high-ranking computational science journals. The aim is to provide an up-to-date picture of current scientific practice with respect to the comparison of methods in both articles presenting new methods and articles focusing on the comparison study itself. Secondly, based on the results of our survey we critically discuss the necessity, impact and limitations of neutral comparison studies in computational sciences. We define three reasonable criteria a comparison study has to fulfill in order to be considered as neutral, and explicate general considerations on the individual components of a "tidy neutral comparison study". R codes for completely replicating our statistical analyses and figures are available from the companion website http://www.ibe.med.uni-muenchen.de/organisation/mitarbeiter/020_professuren/boulesteix/plea2013.

    Original languageEnglish
    Article numbere61562
    Pages (from-to)1-11
    JournalPloS one
    Volume8
    Issue number4
    DOIs
    Publication statusPublished - 24 Apr 2013
    MoE publication typeA1 Journal article-refereed

    Fingerprint Dive into the research topics of 'A Plea for Neutral Comparison Studies in Computational Sciences'. Together they form a unique fingerprint.

    Cite this