Heads-Up Multitasker: Simulating Attention Switching On Optical Head-Mounted Displays

Yunpeng Bai, Aleksi Ikkala, Antti Oulasvirta, Shengdong Zhao, Lucia J Wang, Pengzhi Yang, Peisen Xu

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

17 Downloads (Pure)

Abstract

Optical Head-Mounted Displays (OHMDs) allow users to read digital content while walking. A better understanding of how users allocate attention between these two tasks is crucial for improving OHMD interfaces. This paper introduces a computational model for simulating users’ attention switches between reading and walking. We model users’ decision to deploy visual attention as a hierarchical reinforcement learning problem, wherein a supervisory controller optimizes attention allocation while considering both reading activity and walking safety. Our model simulates the control of eye movements and locomotion as an adaptation to the given task priority, design of digital content, and walking speed. The model replicates key multitasking behaviors during OHMD reading while walking, including attention switches, changes in reading and walking speeds, and reading resumptions.
Original languageEnglish
Title of host publicationCHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
EditorsFlorian Floyd Mueller, Penny Kyburz, Julie R. Williamson, Corina Sas, Max L. Wilson, Phoebe Toups Dugas, Irina Shklovski
PublisherACM
Number of pages18
ISBN (Electronic)979-8-4007-0330-0
DOIs
Publication statusPublished - 11 May 2024
MoE publication typeA4 Conference publication
EventACM SIGCHI Annual Conference on Human Factors in Computing Systems - Honolulu, USA, Honolulu, United States
Duration: 11 May 202416 May 2024
https://chi2024.acm.org/

Conference

ConferenceACM SIGCHI Annual Conference on Human Factors in Computing Systems
Abbreviated titleACM CHI
Country/TerritoryUnited States
CityHonolulu
Period11/05/202416/05/2024
Internet address

Keywords

  • computational rationality
  • multitasking
  • bounded optimal control
  • heads-up computing
  • deep reinforcement learning

Fingerprint

Dive into the research topics of 'Heads-Up Multitasker: Simulating Attention Switching On Optical Head-Mounted Displays'. Together they form a unique fingerprint.

Cite this