Neural Language Models as What If? -Engines for HCI Research

Perttu Hämäläinen; Mikke Tavast; Anton Kunnari

doi:10.1145/3490100.3516458

Neural Language Models as What If? -Engines for HCI Research

Perttu Hämäläinen, Mikke Tavast, Anton Kunnari

University of Helsinki

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

5 Citations (Scopus)

196 Downloads (Pure)

Abstract

Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) and user experience (UX) research. In this poster paper, we explore and critically evaluate the potential of large-scale neural language models like GPT-3 in generating synthetic research data such as participant responses to interview questions. We observe that in the best case, GPT-3 can create plausible reflections of video game experiences and emotions, and adapt its responses to given demographic information. Compared to real participants, such synthetic data can be obtained faster and at a lower cost. On the other hand, the quality of generated data has high variance, and future work is needed to rigorously quantify the human-likeness, limitations, and biases of the models in the HCI domain.

Original language	English
Title of host publication	27th International Conference on Intelligent User Interfaces, IUI 2022 Companion
Publisher	ACM
Pages	77-80
Number of pages	4
ISBN (Electronic)	9781450391450
DOIs	https://doi.org/10.1145/3490100.3516458
Publication status	Published - 22 Mar 2022
MoE publication type	A4 Conference publication
Event	International Conference on Intelligent User Interfaces - Virtual, Online, Finland Duration: 22 Mar 2022 → 25 Mar 2022 Conference number: 27

Publication series

Name	International Conference on Intelligent User Interfaces

Conference

Conference	International Conference on Intelligent User Interfaces
Abbreviated title	IUI
Country/Territory	Finland
City	Virtual, Online
Period	22/03/2022 → 25/03/2022

Keywords

GPT-3
Language models
User experience
User models

Access to Document

10.1145/3490100.3516458

Neural Language Models as What If_Engines for HCI ResearchFinal published version, 422 KB

Cite this

@inproceedings{303e7b1150f349578d0d6f8a0ab40cce,

title = "Neural Language Models as What If? -Engines for HCI Research",

abstract = "Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) and user experience (UX) research. In this poster paper, we explore and critically evaluate the potential of large-scale neural language models like GPT-3 in generating synthetic research data such as participant responses to interview questions. We observe that in the best case, GPT-3 can create plausible reflections of video game experiences and emotions, and adapt its responses to given demographic information. Compared to real participants, such synthetic data can be obtained faster and at a lower cost. On the other hand, the quality of generated data has high variance, and future work is needed to rigorously quantify the human-likeness, limitations, and biases of the models in the HCI domain.",

keywords = "GPT-3, Language models, User experience, User models",

author = "Perttu H{\"a}m{\"a}l{\"a}inen and Mikke Tavast and Anton Kunnari",

note = "Publisher Copyright: {\textcopyright} 2022 Owner/Author.; International Conference on Intelligent User Interfaces, IUI ; Conference date: 22-03-2022 Through 25-03-2022",

year = "2022",

month = mar,

day = "22",

doi = "10.1145/3490100.3516458",

language = "English",

series = "International Conference on Intelligent User Interfaces",

publisher = "ACM",

pages = "77--80",

booktitle = "27th International Conference on Intelligent User Interfaces, IUI 2022 Companion",

address = "United States",

}

Hämäläinen, P , Tavast, M & Kunnari, A 2022, Neural Language Models as What If? -Engines for HCI Research. in 27th International Conference on Intelligent User Interfaces, IUI 2022 Companion. International Conference on Intelligent User Interfaces, ACM, pp. 77-80, International Conference on Intelligent User Interfaces, Virtual, Online, Finland, 22/03/2022. https://doi.org/10.1145/3490100.3516458

Neural Language Models as What If? -Engines for HCI Research. / Hämäläinen, Perttu ; Tavast, Mikke; Kunnari, Anton.
27th International Conference on Intelligent User Interfaces, IUI 2022 Companion. ACM, 2022. p. 77-80 (International Conference on Intelligent User Interfaces).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Neural Language Models as What If? -Engines for HCI Research

AU - Hämäläinen, Perttu

AU - Tavast, Mikke

AU - Kunnari, Anton

N1 - Conference code: 27

PY - 2022/3/22

Y1 - 2022/3/22

N2 - Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) and user experience (UX) research. In this poster paper, we explore and critically evaluate the potential of large-scale neural language models like GPT-3 in generating synthetic research data such as participant responses to interview questions. We observe that in the best case, GPT-3 can create plausible reflections of video game experiences and emotions, and adapt its responses to given demographic information. Compared to real participants, such synthetic data can be obtained faster and at a lower cost. On the other hand, the quality of generated data has high variance, and future work is needed to rigorously quantify the human-likeness, limitations, and biases of the models in the HCI domain.

AB - Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) and user experience (UX) research. In this poster paper, we explore and critically evaluate the potential of large-scale neural language models like GPT-3 in generating synthetic research data such as participant responses to interview questions. We observe that in the best case, GPT-3 can create plausible reflections of video game experiences and emotions, and adapt its responses to given demographic information. Compared to real participants, such synthetic data can be obtained faster and at a lower cost. On the other hand, the quality of generated data has high variance, and future work is needed to rigorously quantify the human-likeness, limitations, and biases of the models in the HCI domain.

KW - GPT-3

KW - Language models

KW - User experience

KW - User models

UR - http://www.scopus.com/inward/record.url?scp=85127735626&partnerID=8YFLogxK

U2 - 10.1145/3490100.3516458

DO - 10.1145/3490100.3516458

M3 - Conference article in proceedings

AN - SCOPUS:85127735626

T3 - International Conference on Intelligent User Interfaces

SP - 77

EP - 80

BT - 27th International Conference on Intelligent User Interfaces, IUI 2022 Companion

PB - ACM

T2 - International Conference on Intelligent User Interfaces

Y2 - 22 March 2022 through 25 March 2022

ER -

Neural Language Models as What If? -Engines for HCI Research

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this