Visual communication in ubiquitous computing: from smartphones to pervasive displays

Maria L. Montoya Freire

Research output: ThesisDoctoral ThesisCollection of Articles

Abstract

Rapid technological advances have enabled the integration of heterogeneous devices into the physical world to provide different kinds of services to users. This led to the emergence of ubiquitous systems that are available at any time and place. A key aspect of such systems is communication, not only among devices but also towards end users. This dissertation investigates several methods to enable visual communication in ubiquitous systems. Specifically, this research addresses visual communication broadly defined in the context of both device-to-device and device-to-people scenarios. In the first case, the main goal is to achieve reliable and effective communication between different devices. To this end, barcodes are leveraged to establish a visual communication channel by using the display and the camera of a smartphone. Accordingly, a mobile application framework is proposed for adaptive and bi-directional communication through barcodes. Moreover, a novel barcode design is devised to provide different data depending on the scanning distance by means of color blending. This novel approach enables new use cases in mobile computing, including casual interactions with public displays and augmented reality for retail. As for the device-to-people scenario, research is carried out to address two important aspects involving how users obtain information from pervasive displays. First, display solutions should be able to reach users and provide meaningful content to them. This can be achieved by extending the coverage of target audience through replication of display content. The same approach allows to monitor the displays to ensure that they are properly functioning and actually showing the intended content. Accordingly, several methods to access display content from different devices are studied and evaluated. Second, user engagement with pervasive displays is particularly challenging as they are easily ignored, especially when they do not convey information relevant to the users. In this context, a novel approach is taken through models suitable to characterize user behavior in observing the content shown on a screen. In particular, the information foraging theory is applied to build several models with the goal of optimizing content selection in tiled display layouts. Accordingly, a display foraging model is devised to predict the time spent by users on a screen as a characterization of their interest. Across several studies, this approach is shown to be effective in characterizing user behavior in real conditions. Finally, inverse foraging models are proposed to infer users' interest from collected data. The models are accurate in predicting interest based on both synthetic and real user data. The obtained results demonstrate that modeling can be effectively applied to improve user engagement in pervasive displays.
Translated title of the contributionVisual communication in ubiquitous computing: from smartphones to pervasive displays
Original languageEnglish
QualificationDoctor's degree
Awarding Institution
  • Aalto University
Supervisors/Advisors
  • Di Francesco, Mario, Supervising Professor
  • Oulasvirta, Antti, Thesis Advisor
Publisher
Print ISBNs978-952-64-0346-5
Electronic ISBNs978-952-64-0347-2
Publication statusPublished - 2021
MoE publication typeG5 Doctoral dissertation (article)

Keywords

  • ubiquitous computing
  • ubiquitous devices
  • visual communication

Fingerprint

Dive into the research topics of 'Visual communication in ubiquitous computing: from smartphones to pervasive displays'. Together they form a unique fingerprint.

Cite this