Real-time 3D Target Inference via Biomechanical Simulation

Hee-Seung Moon, Yi-Chi Liao, Chenyu Li, Byungjoo Lee, Antti Oulasvirta

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

4 Citations (Scopus)
35 Downloads (Pure)

Abstract

Selecting a target in a 3D environment is often challenging, especially with small/distant targets or when sensor noise is high. To facilitate selection, target-inference methods must be accurate, fast, and account for noise and motor variability. However, traditional data-free approaches fall short in accuracy since they ignore variability. While data-driven solutions achieve higher accuracy, they rely on extensive human datasets so prove costly, time-consuming, and transfer poorly. In this paper, we propose a novel approach that leverages biomechanical simulation to produce synthetic motion data, capturing a variety of movement-related factors, such as limb configurations and motor noise. Then, an inference model is trained with only the simulated data. Our simulation-based approach improves transfer and lowers cost; variety-rich data can be produced in large quantities for different scenarios. We empirically demonstrate that our method matches the accuracy of human-data-driven approaches using data from seven users. When deployed, the method accurately infers intended targets in challenging 3D pointing conditions within 5–10 milliseconds, reducing users’ target-selection error by 71% and completion time by 35%.
Original languageEnglish
Title of host publicationCHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
EditorsFlorian Floyd Mueller, Penny Kyburz, Julie R. Williamson, Corina Sas, Max L. Wilson, Phoebe Toups Dugas, Irina Shklovski
PublisherACM
Number of pages18
ISBN (Electronic)979-8-4007-0330-0
DOIs
Publication statusPublished - 11 May 2024
MoE publication typeA4 Conference publication
EventACM SIGCHI Annual Conference on Human Factors in Computing Systems - Honolulu, USA, Honolulu, United States
Duration: 11 May 202416 May 2024
https://chi2024.acm.org/

Conference

ConferenceACM SIGCHI Annual Conference on Human Factors in Computing Systems
Abbreviated titleACM CHI
Country/TerritoryUnited States
CityHonolulu
Period11/05/202416/05/2024
Internet address

Keywords

  • target inference
  • Target selection
  • biomechanical simulation
  • amortized inference

Fingerprint

Dive into the research topics of 'Real-time 3D Target Inference via Biomechanical Simulation'. Together they form a unique fingerprint.

Cite this