CRTypist: Simulating Touchscreen Typing Behavior via Computational Rationality

Danqing Shi, Yujun Zhu, Jussi P.P. Jokinen, Aditya Acharya, Aini Putkonen, Shumin Zhai, Antti Oulasvirta

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

11 Downloads (Pure)

Abstract

Touchscreen typing requires coordinating the fingers and visual attention for button-pressing, proofreading, and error correction. Computational models need to account for the associated fast pace, coordination issues, and closed-loop nature of this control problem, which is further complicated by the immense variety of keyboards and users. The paper introduces CRTypist, which generates human-like typing behavior. Its key feature is a reformulation of the supervisory control problem, with the visual attention and motor system being controlled with reference to a working memory representation tracking the text typed thus far. Movement policy is assumed to asymptotically approach optimal performance in line with cognitive and design-related bounds. This flexible model works directly from pixels, without requiring hand-crafted feature engineering for keyboards. It aligns with human data in terms of movements and performance, covers individual differences, and can generalize to diverse keyboard designs. Though limited to skilled typists, the model generates useful estimates of the typing performance achievable under various conditions.
Original languageEnglish
Title of host publicationCHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
EditorsFlorian Floyd Mueller, Penny Kyburz, Julie R. Williamson, Corina Sas, Max L. Wilson, Phoebe Toups Dugas, Irina Shklovski
PublisherACM
Number of pages17
ISBN (Electronic)979-8-4007-0330-0
DOIs
Publication statusPublished - 11 May 2024
MoE publication typeA4 Conference publication
EventACM SIGCHI Annual Conference on Human Factors in Computing Systems - Honolulu, USA, Honolulu, United States
Duration: 11 May 202416 May 2024
https://chi2024.acm.org/

Conference

ConferenceACM SIGCHI Annual Conference on Human Factors in Computing Systems
Abbreviated titleACM CHI
Country/TerritoryUnited States
CityHonolulu
Period11/05/202416/05/2024
Internet address

Keywords

  • Touchscreen typing
  • Computational modeling
  • Simulation models
  • Reinforcement learning

Fingerprint

Dive into the research topics of 'CRTypist: Simulating Touchscreen Typing Behavior via Computational Rationality'. Together they form a unique fingerprint.

Cite this