Learning to Predict Head Pose in Remotely-Rendered Virtual Reality

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

1 Sitaatiot (Scopus)
46 Lataukset (Pure)


Accurate characterization of Head Mounted Display (HMD) pose in a virtual scene is essential for rendering immersive graphics in Extended Reality (XR). Remote rendering employs servers in the cloud or at the edge of the network to overcome the computational limitations of either standalone or tethered HMDs. Unfortunately, it increases the latency experienced by the user; for this reason, predicting HMD pose in advance is highly beneficial, as long as it achieves high accuracy. This work provides a thorough characterization of solutions that forecast HMD pose in remotely-rendered virtual reality (VR) by considering six degrees of freedom. Specifically, it provides an extensive evaluation of pose representations, forecasting methods, machine learning models, and the use of multiple modalities along with joint and separate training. In particular, a novel three-point representation of pose is introduced together with a data fusion scheme for long-Term short-Term memory (LSTM) neural networks. Our findings show that machine learning models benefit from using multiple modalities, even though simple statistical models perform surprisingly well. Moreover, joint training is comparable to separate training with carefully chosen pose representation and data fusion strategies.

OtsikkoMMSys 2023 - Proceedings of the 14th ACM Multimedia Systems Conference
ISBN (elektroninen)979-8-4007-0148-1
DOI - pysyväislinkit
TilaJulkaistu - 7 kesäk. 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaACM Multimedia Systems Conference - Vancouver, Kanada
Kesto: 7 kesäk. 202310 kesäk. 2023
Konferenssinumero: 14


ConferenceACM Multimedia Systems Conference


Sukella tutkimusaiheisiin 'Learning to Predict Head Pose in Remotely-Rendered Virtual Reality'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä