Learning to Predict Head Pose in Remotely-Rendered Virtual Reality

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

1 Sitaatiot (Scopus)
46 Lataukset (Pure)

Abstrakti

Accurate characterization of Head Mounted Display (HMD) pose in a virtual scene is essential for rendering immersive graphics in Extended Reality (XR). Remote rendering employs servers in the cloud or at the edge of the network to overcome the computational limitations of either standalone or tethered HMDs. Unfortunately, it increases the latency experienced by the user; for this reason, predicting HMD pose in advance is highly beneficial, as long as it achieves high accuracy. This work provides a thorough characterization of solutions that forecast HMD pose in remotely-rendered virtual reality (VR) by considering six degrees of freedom. Specifically, it provides an extensive evaluation of pose representations, forecasting methods, machine learning models, and the use of multiple modalities along with joint and separate training. In particular, a novel three-point representation of pose is introduced together with a data fusion scheme for long-Term short-Term memory (LSTM) neural networks. Our findings show that machine learning models benefit from using multiple modalities, even though simple statistical models perform surprisingly well. Moreover, joint training is comparable to separate training with carefully chosen pose representation and data fusion strategies.

AlkuperäiskieliEnglanti
OtsikkoMMSys 2023 - Proceedings of the 14th ACM Multimedia Systems Conference
KustantajaACM
Sivut27-38
Sivumäärä12
ISBN (elektroninen)979-8-4007-0148-1
DOI - pysyväislinkit
TilaJulkaistu - 7 kesäk. 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaACM Multimedia Systems Conference - Vancouver, Kanada
Kesto: 7 kesäk. 202310 kesäk. 2023
Konferenssinumero: 14

Conference

ConferenceACM Multimedia Systems Conference
LyhennettäMMSys
Maa/AlueKanada
KaupunkiVancouver
Ajanjakso07/06/202310/06/2023

Sormenjälki

Sukella tutkimusaiheisiin 'Learning to Predict Head Pose in Remotely-Rendered Virtual Reality'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä