Vision-to-audio vs Audiovision-to-audio Matching Shows the Difference Between Virtual Acoustics for VR and AR

Virtual reality (VR) experiences immerse users in a virtual world distinct from their own environment. In contrast, augmented reality (AR) applications, such as AR telepresence, add virtual objects to the users’ real environment through semi-transparent displays or visually occluding devices with camera-based passthrough. At first sight, the necessary virtual acoustic technologies for both applications appear similar—position and orientation-dependent binaural room acoustic rendering. However, the perceptual demands on the rendering systems differ considerably. In VR, users can only compare an acoustic rendering to the visual scene they are transported to. In AR, however, real-world sound sources may coexist with virtual ones, allowing for auditory comparison between real and virtual sound. This study compares subjects’ ability to match binaural speech rendering using different rooms to a given reference room presented visually, through another sound source rendered in it, or through both. Therein, the hypothesis is that subjects will be better at matching when an auditory reference is provided. This experiment illustrates a critical distinction between acoustic rendering for VR and AR.