Understanding visual saliency in mobile user interfaces

Luis A. Leiva; Yunfei Xue; Avya Bansal; Hamed R. Tavakoli; Tuðçe Köroðlu; Jingzhou Du; Niraj R. Dayama; Antti Oulasvirta

doi:10.1145/3379503.3403557

Understanding visual saliency in mobile user interfaces

Luis A. Leiva, Yunfei Xue, Avya Bansal, Hamed R. Tavakoli, Tuðçe Köroðlu, Jingzhou Du, Niraj R. Dayama, Antti Oulasvirta

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

14 Citations (Scopus)

290 Downloads (Pure)

Abstract

For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in several respects. We present findings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of expectations in guiding where users look at. Strong bias toward the top-left corner of the display, text, and images was evident, while bottom-up features such as color or size affected saliency less. Classic, parameter-free saliency models showed a weak fit with the data, and data-driven models improved significantly when trained specifically on this dataset (e.g., NSS rose from 0.66 to 0.84). We also release the first annotated dataset for investigating visual saliency in mobile UIs.

Original language	English
Title of host publication	Conference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services
Subtitle of host publication	Expanding the Horizon of Mobile Interaction, MobileHCI 2020
Publisher	ACM
Number of pages	12
ISBN (Electronic)	9781450375160
DOIs	https://doi.org/10.1145/3379503.3403557
Publication status	Published - 5 Oct 2020
MoE publication type	A4 Conference publication
Event	International Conference on Human-Computer Interaction with Mobile Devices and Services - Oldenburg, Germany Duration: 5 Oct 2020 → 9 Oct 2020 Conference number: 22

Conference

Conference	International Conference on Human-Computer Interaction with Mobile Devices and Services
Abbreviated title	MobileHCI
Country/Territory	Germany
City	Oldenburg
Period	05/10/2020 → 09/10/2020

Keywords

Computer Vision
Deep Learning
Human Perception and Cognition
Interaction Design

Access to Document

10.1145/3379503.3403557

Leiva_Understanding_visual_saliency_in_Mobile_UIFinal published version, 14.1 MB

Cite this

Leiva, L. A., Xue, Y., Bansal, A., Tavakoli, H. R., Köroðlu, T., Du, J., Dayama, N. R., & Oulasvirta, A. (2020). Understanding visual saliency in mobile user interfaces. In Conference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services: Expanding the Horizon of Mobile Interaction, MobileHCI 2020 Article 3403557 ACM. https://doi.org/10.1145/3379503.3403557

@inproceedings{ec50f457699646bb94c5a25467be7a1e,

title = "Understanding visual saliency in mobile user interfaces",

abstract = "For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in several respects. We present findings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of expectations in guiding where users look at. Strong bias toward the top-left corner of the display, text, and images was evident, while bottom-up features such as color or size affected saliency less. Classic, parameter-free saliency models showed a weak fit with the data, and data-driven models improved significantly when trained specifically on this dataset (e.g., NSS rose from 0.66 to 0.84). We also release the first annotated dataset for investigating visual saliency in mobile UIs.",

keywords = "Computer Vision, Deep Learning, Human Perception and Cognition, Interaction Design",

author = "Leiva, {Luis A.} and Yunfei Xue and Avya Bansal and Tavakoli, {Hamed R.} and Tu{\dh}{\c c}e K{\"o}ro{\dh}lu and Jingzhou Du and Dayama, {Niraj R.} and Antti Oulasvirta",

note = "| openaire: EC/H2020/637991/EU//COMPUTED; International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ; Conference date: 05-10-2020 Through 09-10-2020",

year = "2020",

month = oct,

day = "5",

doi = "10.1145/3379503.3403557",

language = "English",

booktitle = "Conference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services",

publisher = "ACM",

address = "United States",

}

Leiva, LA, Xue, Y, Bansal, A, Tavakoli, HR, Köroðlu, T, Du, J, Dayama, NR & Oulasvirta, A 2020, Understanding visual saliency in mobile user interfaces. in Conference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services: Expanding the Horizon of Mobile Interaction, MobileHCI 2020., 3403557, ACM, International Conference on Human-Computer Interaction with Mobile Devices and Services, Oldenburg, Germany, 05/10/2020. https://doi.org/10.1145/3379503.3403557

Understanding visual saliency in mobile user interfaces. / Leiva, Luis A.; Xue, Yunfei; Bansal, Avya et al.
Conference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services: Expanding the Horizon of Mobile Interaction, MobileHCI 2020. ACM, 2020. 3403557.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Understanding visual saliency in mobile user interfaces

AU - Leiva, Luis A.

AU - Xue, Yunfei

AU - Bansal, Avya

AU - Tavakoli, Hamed R.

AU - Köroðlu, Tuðçe

AU - Du, Jingzhou

AU - Dayama, Niraj R.

AU - Oulasvirta, Antti

N1 - Conference code: 22

PY - 2020/10/5

Y1 - 2020/10/5

N2 - For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in several respects. We present findings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of expectations in guiding where users look at. Strong bias toward the top-left corner of the display, text, and images was evident, while bottom-up features such as color or size affected saliency less. Classic, parameter-free saliency models showed a weak fit with the data, and data-driven models improved significantly when trained specifically on this dataset (e.g., NSS rose from 0.66 to 0.84). We also release the first annotated dataset for investigating visual saliency in mobile UIs.

AB - For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in several respects. We present findings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of expectations in guiding where users look at. Strong bias toward the top-left corner of the display, text, and images was evident, while bottom-up features such as color or size affected saliency less. Classic, parameter-free saliency models showed a weak fit with the data, and data-driven models improved significantly when trained specifically on this dataset (e.g., NSS rose from 0.66 to 0.84). We also release the first annotated dataset for investigating visual saliency in mobile UIs.

KW - Computer Vision

KW - Deep Learning

KW - Human Perception and Cognition

KW - Interaction Design

UR - http://www.scopus.com/inward/record.url?scp=85094970972&partnerID=8YFLogxK

U2 - 10.1145/3379503.3403557

DO - 10.1145/3379503.3403557

M3 - Conference article in proceedings

AN - SCOPUS:85094970972

BT - Conference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services

PB - ACM

T2 - International Conference on Human-Computer Interaction with Mobile Devices and Services

Y2 - 5 October 2020 through 9 October 2020

ER -

Understanding visual saliency in mobile user interfaces

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

-: Bayesian Artefact Design

COMPUTED: Computational User Interface Design

Science-IT

Cite this

Understanding visual saliency in mobile user interfaces

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

-: Bayesian Artefact Design

COMPUTED: Computational User Interface Design

Equipment

Science-IT

Cite this