Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Rishabh Gupta; Jianjun He; Rishabh Ranjan; Woon Seng Gan; Florian Klein; Christian Schneiderwind; Annika Neidhardt; Karlheinz Brandenburg; Vesa Valimaki

doi:10.1109/MSP.2021.3110108

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Rishabh Gupta, Jianjun He, Rishabh Ranjan, Woon Seng Gan, Florian Klein, Christian Schneiderwind, Annika Neidhardt, Karlheinz Brandenburg, Vesa Valimaki

Audio Signal Processing

Research output: Contribution to journal › Review Article › peer-review

25 Citations (Scopus)

527 Downloads (Pure)

Abstract

Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.

Original language	English
Pages (from-to)	63-89
Number of pages	27
Journal	IEEE Signal Processing Magazine
Volume	39
Issue number	3
DOIs	https://doi.org/10.1109/MSP.2021.3110108
Publication status	Published - 1 May 2022
MoE publication type	A2 Review article, Literature review, Systematic review

Access to Document

10.1109/MSP.2021.3110108

Gupta_Augmented_reality_audio_IEEE_SPMAccepted author manuscript, 10.6 MB

Cite this

@article{b386d099dbaf4309ba120e7d4997d328,

title = "Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering",

abstract = "Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.",

author = "Rishabh Gupta and Jianjun He and Rishabh Ranjan and Gan, {Woon Seng} and Florian Klein and Christian Schneiderwind and Annika Neidhardt and Karlheinz Brandenburg and Vesa Valimaki",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.",

year = "2022",

month = may,

day = "1",

doi = "10.1109/MSP.2021.3110108",

language = "English",

volume = "39",

pages = "63--89",

journal = "IEEE Signal Processing Magazine",

issn = "1053-5888",

publisher = "IEEE",

number = "3",

}

TY - JOUR

T1 - Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

AU - Gupta, Rishabh

AU - He, Jianjun

AU - Ranjan, Rishabh

AU - Gan, Woon Seng

AU - Klein, Florian

AU - Schneiderwind, Christian

AU - Neidhardt, Annika

AU - Brandenburg, Karlheinz

AU - Valimaki, Vesa

PY - 2022/5/1

Y1 - 2022/5/1

N2 - Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.

AB - Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.

UR - http://www.scopus.com/inward/record.url?scp=85130166703&partnerID=8YFLogxK

U2 - 10.1109/MSP.2021.3110108

DO - 10.1109/MSP.2021.3110108

M3 - Review Article

AN - SCOPUS:85130166703

SN - 1053-5888

VL - 39

SP - 63

EP - 89

JO - IEEE Signal Processing Magazine

JF - IEEE Signal Processing Magazine

IS - 3

ER -

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Abstract

Access to Document

Other files and links

Fingerprint

NordicSMC: Nordic Sound and Music Computing Network

Aalto Acoustics Lab

Cite this

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Abstract

Access to Document

Other files and links

Fingerprint

Projects

NordicSMC: Nordic Sound and Music Computing Network

Equipment

Aalto Acoustics Lab

Cite this