Learning GUI Completions with User-defined Constraints

Lukas Brückner, Luis A. Leiva, Antti Oulasvirta

Tutkimustuotos: LehtiartikkeliArticleScientificvertaisarvioitu

45 Lataukset (Pure)


A key objective in the design of graphical user interfaces (GUIs) is to ensure consistency across screens of the same product. However, designing a compliant layout is time-consuming and can distract designers from creative thinking. This paper studies layout recommendation methods that fulfill such consistency requirements using machine learning. Given a desired element type and size, the methods suggest element placements following real-world GUI design processes. Consistency requirements are given implicitly through previous layouts from which patterns are to be learned, comparable to existing screens of a software product. We adopt two recently proposed methods for this task, a Graph Neural Network (GNN) and a Transformer model, and compare them with a custom approach based on sequence alignment and nearest neighbor search (kNN). The methods were tested on handcrafted datasets with explicit layout patterns, as well as large-scale public datasets of diverse mobile design layouts. Our results show that our instance-based learning algorithm outperforms both neural network approaches. Ultimately, this work contributes to establishing smarter design tools for professional designers with explainable algorithms that increase their efficacy.

JulkaisuACM Transactions on Interactive Intelligent Systems
DOI - pysyväislinkit
TilaJulkaistu - maalisk. 2022
OKM-julkaisutyyppiA1 Julkaistu artikkeli, soviteltu


Sukella tutkimusaiheisiin 'Learning GUI Completions with User-defined Constraints'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä