WigglyEyes: Inferring Eye Movements from Keypress Data

Yujun Zhu; Danqing Shi; Hee-Seung Moon; Antti Oulasvirta

doi:10.1145/3715071.3750430

WigglyEyes: Inferring Eye Movements from Keypress Data

Yujun Zhu, Danqing Shi^*, Hee-Seung Moon, Antti Oulasvirta

^*Corresponding author for this work

Chung-Ang University

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

1 Downloads (Pure)

Abstract

We present a model for inferring where users look during interaction based on keypress data only. Given a key log, it outputs a scanpath that tells, moment-by-moment, how the user had moved eyes while entering those keys. The model can be used as a proxy for human data in cases where collecting real eye tracking data is expensive or impossible. Our technical insight is an inference architecture that considers the individual characteristics of the user, inferred as a low-dimensional parameter vector. We present a novel loss function for synchronizing inferred eye movements with the keypresses. Evaluations on touchscreen typing demonstrate accurate gaze inference.

Original language	English
Title of host publication	ISWC 2025 - Proceedings of the 2025 ACM International Symposium on Wearable Computers
Publisher	ACM
Pages	191-194
Number of pages	4
ISBN (Electronic)	979-8-4007-1481-8
DOIs	https://doi.org/10.1145/3715071.3750430
Publication status	Published - 7 Oct 2025
MoE publication type	A4 Conference publication
Event	International Symposium on Wearable Computers - Espoo, Finland Duration: 12 Oct 2025 → 16 Oct 2025 Conference number: 29

Conference

Conference	International Symposium on Wearable Computers
Abbreviated title	ISWC
Country/Territory	Finland
City	Espoo
Period	12/10/2025 → 16/10/2025

Funding

This paper is supported by the Research Council of Finland (FCAI: 328400, 345604, 341763; Subjective Functions 357578), the ERC (AdG project Artificial User: 101141916.), and Google Grant (DeepTypist).

Keywords

eye movement
scanpath prediction
text entry
user model

Access to Document

10.1145/3715071.3750430Licence: CC BY

WigglyEyes_Inferring_Eye_Movements_from_Keypress_DataFinal published version, 1.81 MBLicence: CC BY

Cite this

@inproceedings{483507707acf4dcaa80f78cb82a8875d,

title = "WigglyEyes: Inferring Eye Movements from Keypress Data",

abstract = "We present a model for inferring where users look during interaction based on keypress data only. Given a key log, it outputs a scanpath that tells, moment-by-moment, how the user had moved eyes while entering those keys. The model can be used as a proxy for human data in cases where collecting real eye tracking data is expensive or impossible. Our technical insight is an inference architecture that considers the individual characteristics of the user, inferred as a low-dimensional parameter vector. We present a novel loss function for synchronizing inferred eye movements with the keypresses. Evaluations on touchscreen typing demonstrate accurate gaze inference.",

keywords = "eye movement, scanpath prediction, text entry, user model",

author = "Yujun Zhu and Danqing Shi and Hee-Seung Moon and Antti Oulasvirta",

year = "2025",

month = oct,

day = "7",

doi = "10.1145/3715071.3750430",

language = "English",

pages = "191--194",

booktitle = "ISWC 2025 - Proceedings of the 2025 ACM International Symposium on Wearable Computers",

publisher = "ACM",

address = "United States",

note = "International Symposium on Wearable Computers, ISWC ; Conference date: 12-10-2025 Through 16-10-2025",

}

TY - GEN

T1 - WigglyEyes: Inferring Eye Movements from Keypress Data

AU - Zhu, Yujun

AU - Shi, Danqing

AU - Moon, Hee-Seung

AU - Oulasvirta, Antti

N1 - Conference code: 29

PY - 2025/10/7

Y1 - 2025/10/7

N2 - We present a model for inferring where users look during interaction based on keypress data only. Given a key log, it outputs a scanpath that tells, moment-by-moment, how the user had moved eyes while entering those keys. The model can be used as a proxy for human data in cases where collecting real eye tracking data is expensive or impossible. Our technical insight is an inference architecture that considers the individual characteristics of the user, inferred as a low-dimensional parameter vector. We present a novel loss function for synchronizing inferred eye movements with the keypresses. Evaluations on touchscreen typing demonstrate accurate gaze inference.

AB - We present a model for inferring where users look during interaction based on keypress data only. Given a key log, it outputs a scanpath that tells, moment-by-moment, how the user had moved eyes while entering those keys. The model can be used as a proxy for human data in cases where collecting real eye tracking data is expensive or impossible. Our technical insight is an inference architecture that considers the individual characteristics of the user, inferred as a low-dimensional parameter vector. We present a novel loss function for synchronizing inferred eye movements with the keypresses. Evaluations on touchscreen typing demonstrate accurate gaze inference.

KW - eye movement

KW - scanpath prediction

KW - text entry

KW - user model

UR - https://www.scopus.com/pages/publications/105021378511

U2 - 10.1145/3715071.3750430

DO - 10.1145/3715071.3750430

M3 - Conference article in proceedings

SP - 191

EP - 194

BT - ISWC 2025 - Proceedings of the 2025 ACM International Symposium on Wearable Computers

PB - ACM

T2 - International Symposium on Wearable Computers

Y2 - 12 October 2025 through 16 October 2025

ER -

WigglyEyes: Inferring Eye Movements from Keypress Data

Abstract

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this