Modeling Gliding-based Target Selection for Blind Touchscreen Users

Yu Jung Ko; Aini Putkonen; Ali Selman Aydin; Shirin Feiz; Yuheng Wang; Vikas Ashok; IV Ramakrishnan; Antti Oulasvirta; Xiaojun Bi

doi:10.1145/3447526.3472022

Modeling Gliding-based Target Selection for Blind Touchscreen Users

Yu Jung Ko, Aini Putkonen, Ali Selman Aydin, Shirin Feiz, Yuheng Wang, Vikas Ashok, IV Ramakrishnan, Antti Oulasvirta, Xiaojun Bi

User Interfaces

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

1 Citation (Scopus)

223 Downloads (Pure)

Abstract

Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements with iterative corrections relying on non-visual feedback, and 2) multiple sub-movements separated by stops, and concatenated until the target is reached. Based on this finding, we propose the mixture pointing model, a model that relates movement time to distance and width of the target. The model outperforms extant models, improving R2 from 0.65 for Fitts' law to 0.76, and is superior in cross-validation and information criteria. The model advances understanding of gliding-based target selection and serves as a tool for designing interface layouts for screen-reader based touch exploration.

Original language	English
Title of host publication	MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interactionn
Subtitle of host publication	Mobile Apart, MobileTogether
Publisher	ACM
Pages	1-14
Number of pages	14
ISBN (Electronic)	9781450383288
DOIs	https://doi.org/10.1145/3447526.3472022
Publication status	Published - 27 Sept 2021
MoE publication type	A4 Conference publication
Event	International Conference on Human-Computer Interaction with Mobile Devices and Services - Virtual, Online, France Duration: 27 Sept 2021 → 1 Oct 2021 Conference number: 23

Conference

Conference	International Conference on Human-Computer Interaction with Mobile Devices and Services
Abbreviated title	MobileHCI
Country/Territory	France
City	Virtual, Online
Period	27/09/2021 → 01/10/2021

Keywords

accessibility
gliding-based target selection
pointing

Access to Document

10.1145/3447526.3472022

ELEC_Ko_etal_Modeling_Gliding_Based_Target_Selection_MobileHCI_2021Accepted author manuscript, 5.64 MB

Cite this

Ko, Y. J., Putkonen, A., Aydin, A. S., Feiz, S., Wang, Y., Ashok, V., Ramakrishnan, IV., Oulasvirta, A., & Bi, X. (2021). Modeling Gliding-based Target Selection for Blind Touchscreen Users. In MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interactionn: Mobile Apart, MobileTogether (pp. 1-14). Article 29 ACM. https://doi.org/10.1145/3447526.3472022

@inproceedings{cbf6c2c9a734424a940ed034112e2cc8,

title = "Modeling Gliding-based Target Selection for Blind Touchscreen Users",

abstract = "Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements with iterative corrections relying on non-visual feedback, and 2) multiple sub-movements separated by stops, and concatenated until the target is reached. Based on this finding, we propose the mixture pointing model, a model that relates movement time to distance and width of the target. The model outperforms extant models, improving R2 from 0.65 for Fitts' law to 0.76, and is superior in cross-validation and information criteria. The model advances understanding of gliding-based target selection and serves as a tool for designing interface layouts for screen-reader based touch exploration. ",

keywords = "accessibility, gliding-based target selection, pointing",

author = "Ko, {Yu Jung} and Aini Putkonen and Aydin, {Ali Selman} and Shirin Feiz and Yuheng Wang and Vikas Ashok and IV Ramakrishnan and Antti Oulasvirta and Xiaojun Bi",

note = "Funding Information: We thank the anonymous reviewers for their insightful comments. This work was supported by NSF awards 1815514, 1805076, 1936027, NIH awards R01EY030085 and R01HD097188. AO and AP were funded by the Finnish Center for Artificial Intelligence FCAI, and the Academy of Finland projects Human Automata (Project ID: 328813) and BAD (Project ID: 318559). Publisher Copyright: {\textcopyright} 2021 ACM.; International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ; Conference date: 27-09-2021 Through 01-10-2021",

year = "2021",

month = sep,

day = "27",

doi = "10.1145/3447526.3472022",

language = "English",

pages = "1--14",

booktitle = "MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interactionn",

publisher = "ACM",

address = "United States",

}

Ko, YJ, Putkonen, A, Aydin, AS, Feiz, S, Wang, Y, Ashok, V, Ramakrishnan, IV, Oulasvirta, A & Bi, X 2021, Modeling Gliding-based Target Selection for Blind Touchscreen Users. in MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interactionn: Mobile Apart, MobileTogether., 29, ACM, pp. 1-14, International Conference on Human-Computer Interaction with Mobile Devices and Services, Virtual, Online, France, 27/09/2021. https://doi.org/10.1145/3447526.3472022

Modeling Gliding-based Target Selection for Blind Touchscreen Users. / Ko, Yu Jung; Putkonen, Aini; Aydin, Ali Selman et al.
MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interactionn: Mobile Apart, MobileTogether. ACM, 2021. p. 1-14 29.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Modeling Gliding-based Target Selection for Blind Touchscreen Users

AU - Ko, Yu Jung

AU - Putkonen, Aini

AU - Aydin, Ali Selman

AU - Feiz, Shirin

AU - Wang, Yuheng

AU - Ashok, Vikas

AU - Ramakrishnan, IV

AU - Oulasvirta, Antti

AU - Bi, Xiaojun

N1 - Conference code: 23

PY - 2021/9/27

Y1 - 2021/9/27

N2 - Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements with iterative corrections relying on non-visual feedback, and 2) multiple sub-movements separated by stops, and concatenated until the target is reached. Based on this finding, we propose the mixture pointing model, a model that relates movement time to distance and width of the target. The model outperforms extant models, improving R2 from 0.65 for Fitts' law to 0.76, and is superior in cross-validation and information criteria. The model advances understanding of gliding-based target selection and serves as a tool for designing interface layouts for screen-reader based touch exploration.

AB - Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements with iterative corrections relying on non-visual feedback, and 2) multiple sub-movements separated by stops, and concatenated until the target is reached. Based on this finding, we propose the mixture pointing model, a model that relates movement time to distance and width of the target. The model outperforms extant models, improving R2 from 0.65 for Fitts' law to 0.76, and is superior in cross-validation and information criteria. The model advances understanding of gliding-based target selection and serves as a tool for designing interface layouts for screen-reader based touch exploration.

KW - accessibility

KW - gliding-based target selection

KW - pointing

UR - http://www.scopus.com/inward/record.url?scp=85117312202&partnerID=8YFLogxK

U2 - 10.1145/3447526.3472022

DO - 10.1145/3447526.3472022

M3 - Conference article in proceedings

AN - SCOPUS:85117312202

SP - 1

EP - 14

BT - MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interactionn

PB - ACM

T2 - International Conference on Human-Computer Interaction with Mobile Devices and Services

Y2 - 27 September 2021 through 1 October 2021

ER -

Modeling Gliding-based Target Selection for Blind Touchscreen Users

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Human Automata: Simulator-based Methods for Collaborative AI

-: Finnish Center for Artificial Intelligence

-: Bayesian Artefact Design

Cite this

Modeling Gliding-based Target Selection for Blind Touchscreen Users

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Human Automata: Simulator-based Methods for Collaborative AI

-: Finnish Center for Artificial Intelligence

-: Bayesian Artefact Design

Cite this