InDepth: Real-time Depth Inpainting for Mobile Augmented Reality

Yunfan Zhang, Tim Scargill, Ashutosh Vaishnav, Gopika Premsankar, Mario Di Francesco, Maria Gorlatova

Tutkimustuotos: LehtiartikkeliArticleScientificvertaisarvioitu

11 Sitaatiot (Scopus)
197 Lataukset (Pure)

Abstrakti

Mobile Augmented Reality (AR) demands realistic rendering of virtual content that seamlessly blends into the physical environment. For this reason, AR headsets and recent smartphones are increasingly equipped with Time-of-Flight (ToF) cameras to acquire depth maps of a scene in real-time. ToF cameras are cheap and fast, however, they suffer from several issues that affect the quality of depth data, ultimately hampering their use for mobile AR. Among them, scale errors of virtual objects - appearing much bigger or smaller than what they should be - are particularly noticeable and unpleasant. This article specifically addresses these challenges by proposing InDepth, a real-time depth inpainting system based on edge computing. InDepth employs a novel deep neural network (DNN) architecture to improve the accuracy of depth maps obtained from ToF cameras. The DNN fills holes and corrects artifacts in the depth maps with high accuracy and eight times lower inference time than the state of the art. An extensive performance evaluation in real settings shows that InDepth reduces the mean absolute error by a factor of four with respect to ARCore DepthLab. Finally, a user study reveals that InDepth is effective in rendering correctly-scaled virtual objects, outperforming DepthLab.

AlkuperäiskieliEnglanti
Artikkeli37
Sivut1-25
Sivumäärä25
JulkaisuProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Vuosikerta6
Numero1
DOI - pysyväislinkit
TilaJulkaistu - maalisk. 2022
OKM-julkaisutyyppiA1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä

Sormenjälki

Sukella tutkimusaiheisiin 'InDepth: Real-time Depth Inpainting for Mobile Augmented Reality'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä