VIMES: A Wearable Memory Assistance System for Automatic Information Retrieval

Carlos Bermejo, Tristan Braud, Ji Yang, Shayan Mirjafari, Bowen Shi, Yu Xiao, Pan Hui

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

The advancement of artificial intelligence and wearable computing triggers the radical innovation of cognitive applications. In this work, we propose VIMES, an augmented reality-based memory assistance system that helps recall declarative memory, such as whom the user meets and what they chat. Through a collaborative method with 20 participants, we design VIMES, a system that runs on smartglasses, takes the first-person audio and video as input, and extracts personal profiles and event information to display on the embedded display or a smartphone. We perform an extensive evaluation with 50 participants to show the effectiveness of VIMES for memory recall. VIMES outperforms (90% memory accuracy) other traditional methods such as self-recall (34%) while offering the best memory experience (Vividness, Coherence, and Visual Perspective all score over 4/5). The user study results show that most participants find VIMES useful (3.75/5) and easy to use (3.46/5).
Original languageEnglish
Title of host publicationProceedings of the 28th ACM International Conference on Multimedia
PublisherACM
Pages3191-3200
Number of pages10
ISBN (Electronic)978-1-4503-7988-5
DOIs
Publication statusPublished - Oct 2020
MoE publication typeA4 Article in a conference publication
EventACM International Conference on Multimedia - Virtual, Online
Duration: 12 Oct 202016 Oct 2020
Conference number: 28

Conference

ConferenceACM International Conference on Multimedia
Abbreviated titleMM
CityVirtual, Online
Period12/10/202016/10/2020

Keywords

  • information retrieval
  • wearable computing
  • memory assistance system

Fingerprint

Dive into the research topics of 'VIMES: A Wearable Memory Assistance System for Automatic Information Retrieval'. Together they form a unique fingerprint.

Cite this