Developing hand-worn input and haptic support for real-world target finding

Yi Ta Hsieh*, Antti Jylhä, Valeria Orso, Salvatore Andolina, Eve Hoggan, Luciano Gamberini, Giulio Jacucci

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)


Locating places in cities is typically facilitated by handheld mobile devices, which draw the visual attention of the user on the screen of the device instead of the surroundings. In this research, we aim at strengthening the connection between people and their surroundings through enabling mid-air gestural interaction with real-world landmarks and delivering information through audio to retain users’ visual attention on the scene. Recent research on gesture-based and haptic techniques for such purposes has mainly considered handheld devices that eventually direct users’ attention back to the devices. We contribute a hand-worn, mid-air gestural interaction design with directional vibrotactile guidance for finding points of interest (POIs). Through three design iterations, we address aspects of (1) sensing technologies and the placement of actuators considering users’ instinctive postures, (2) the feasibility of finding and fetching information regarding landmarks without visual feedback, and (3) the benefits of such interaction in a tourist application. In a final evaluation, participants located POIs and fetched information by pointing and following directional guidance, thus realising a vision in which they found and experienced real-world landmarks while keeping their visual attention on the scene. The results show that the interaction technique has comparable performance to a visual baseline, enables high mobility, and facilitates keeping visual attention on the surroundings.

Original languageEnglish
Pages (from-to)117–132
Number of pages16
JournalPersonal and Ubiquitous Computing
Issue number1
Early online date1 Jan 2018
Publication statusPublished - Feb 2019
MoE publication typeA1 Journal article-refereed


  • Auditory feedback
  • Augmented reality
  • Gestural input
  • Haptic devices
  • Novel interaction paradigms
  • Pointing
  • Ubiquitous and mobile computing design and evaluation


Dive into the research topics of 'Developing hand-worn input and haptic support for real-world target finding'. Together they form a unique fingerprint.

Cite this