Conversations with GUIs

Kashyap Todi, Luis A. Leiva, Daniel Buschek, Pin Tian, Antti Oulasvirta

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

5 Sitaatiot (Scopus)
150 Lataukset (Pure)

Abstrakti

Annotated datasets of application GUIs contain a wealth of information that can be used for various purposes, from providing inspiration to designers and implementation details to developers to assisting end-users during daily use. However, users often struggle to formulate their needs in a way that computers can understand reliably. To address this, we study how people may interact with such GUI datasets using natural language. We elicit user needs in a survey (N = 120) with three target groups (designers, developers, end-users), providing insights into which capabilities would be useful and how users formulate queries. We contribute a labelled dataset of 1317 user queries, and demonstrate an application of a conversational assistant that interprets these queries and retrieves information from a large-scale GUI dataset. It can (1) suggest GUI screenshots for design ideation, (2) highlight details about particular GUI features for development, and (3) reveal further insights about applications. Our fndings can inform design and implementation of intelligent systems to interact with GUI datasets intuitively.

AlkuperäiskieliEnglanti
OtsikkoDIS 2021 - Proceedings of the 2021 ACM Designing Interactive Systems Conference
AlaotsikkoNowhere and Everywhere
KustantajaACM
Sivut1447-1457
Sivumäärä11
ISBN (elektroninen)9781450384766
DOI - pysyväislinkit
TilaJulkaistu - 28 kesäk. 2021
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaACM Conference on Designing Interactive Systems: Nowhere and Everywhere - Virtual, Online, Yhdysvallat
Kesto: 28 kesäk. 20212 heinäk. 2021

Conference

ConferenceACM Conference on Designing Interactive Systems
LyhennettäDIS
Maa/AlueYhdysvallat
KaupunkiVirtual, Online
Ajanjakso28/06/202102/07/2021

Sormenjälki

Sukella tutkimusaiheisiin 'Conversations with GUIs'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä