Breathing Life Into Biomechanical User Models

Aleksi Ikkala, Florian Fischer, Markus Klar, Miroslav Bachinski, Arthur Fleig, Andrew Howes, Perttu Hämäläinen, Jörg Müller, Roderick Murray-Smith, Antti Oulasvirta

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

35 Downloads (Pure)


Forward biomechanical simulation in HCI holds great promise as a tool for evaluation, design, and engineering of user interfaces. Although reinforcement learning (RL) has been used to simulate biomechanics in interaction, prior work has relied on unrealistic assumptions about the control problem involved, which limits the plausibility of emerging policies. These assumptions include direct torque actuation as opposed to muscle-based control; direct, privileged access to the external environment, instead of imperfect sensory observations; and lack of interaction with physical input devices. In this paper, we present a new approach for learning muscle-actuated control policies based on perceptual feedback in interaction tasks with physical input devices. This allows modelling of more realistic interaction tasks with cognitively plausible visuomotor control. We show that our simulated user model successfully learns a variety of tasks representing different interaction methods, and that the model exhibits characteristic movement regularities observed in studies of pointing. We provide an open-source implementation which can be extended with further biomechanical models, perception models, and interactive environments.
Original languageEnglish
Title of host publicationUIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology
Number of pages14
ISBN (Electronic)978-1-4503-9320-1
Publication statusPublished - 28 Oct 2022
MoE publication typeA4 Article in a conference publication
EventACM Symposium on User Interface Software and Technology - Bend, United States
Duration: 29 Oct 20222 Nov 2022
Conference number: 35


ConferenceACM Symposium on User Interface Software and Technology
Abbreviated titleUIST
Country/TerritoryUnited States


Dive into the research topics of 'Breathing Life Into Biomechanical User Models'. Together they form a unique fingerprint.

Cite this