Statistical Significance Testing at CHI PLAY: Challenges and Opportunities for More Transparency

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

18 Citations (Scopus)
179 Downloads (Pure)

Abstract

Statistical Significance Testing -- or Null Hypothesis Significance Testing (NHST) -- is common to quantitative CHI PLAY research. Drawing from recent work in HCI and psychology promoting transparent statistics and the reduction of questionable research practices, we systematically review the reporting quality of 119 CHI PLAY papers using NHST (data and analysis plan at https://osf.io/4mcbn/. We find that over half of these papers employ NHST without specific statistical hypotheses or research questions, which may risk the proliferation of false positive findings. Moreover, we observe inconsistencies in the reporting of sample sizes and statistical tests. These issues reflect fundamental incompatibilities between NHST and the frequently exploratory work common to CHI PLAY. We discuss the complementary roles of exploratory and confirmatory research, and provide a template for more transparent research and reporting practices.
Original languageEnglish
Title of host publicationCHI PLAY 2020 - Proceedings of the Annual Symposium on Computer-Human Interaction in Play
PublisherACM
Pages4-18
Number of pages15
ISBN (Electronic)978-1-4503-8074-4
DOIs
Publication statusPublished - 2 Nov 2020
MoE publication typeA4 Article in a conference publication
EventACM SIGCHI Annual Symposium on Computer-Human Interaction in Play - Virtual, Online, Canada
Duration: 1 Nov 20204 Nov 2020
Conference number: 7

Conference

ConferenceACM SIGCHI Annual Symposium on Computer-Human Interaction in Play
Abbreviated titleCHI PLAY
Country/TerritoryCanada
CityVirtual, Online
Period01/11/202004/11/2020

Fingerprint

Dive into the research topics of 'Statistical Significance Testing at CHI PLAY: Challenges and Opportunities for More Transparency'. Together they form a unique fingerprint.

Cite this