Functional Gaze Prediction in Egocentric Video

Si-Ahmed Naas, Xiaolan Jiang, Stephan Sigg, Yusheng Ji

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

148 Lataukset (Pure)

Abstrakti

Streaming 360 videos to a head-mounted display (HMD) client is challenging due to their high network resource consumption and computational load. This is due to the use of gaze point prediction or image saliency features from the field of view (FoV) since, in real-time scenarios, FoV extraction is computationally demanding. We propose a functional gaze prediction system that addresses these issues by relying on a tiling scheme for gaze prediction. We condition gaze point prediction on virtual reality (VR) content and long short-term memory (LSTM)-encoded eye movement history. Further, we encode image flow and saliency maps of RGB images via VGG16, using a convolutional neural network (CNN). Future gaze points are then predicted using a novel sinusoidal encoding technique. In experiments, our tile-based approach outperforms state-of-the-art FoV-based schemes in terms of computational load and predicted gaze position.
AlkuperäiskieliEnglanti
Otsikko18th International Conference on Advances in Mobile Computing and Multimedia, MoMM2020 - Proceedings
ToimittajatPari Delir Haghighi, Ivan Luiz Salvadori, Matthias Steinbauer, Ismail Khalil, Gabriele Kotsis
KustantajaACM
Sivut40-47
Sivumäärä8
ISBN (elektroninen)9781450389242
DOI - pysyväislinkit
TilaJulkaistu - 30 marrask. 2020
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Conference on Advances in Mobile Computing & Multimedia - Chiang Mai, Thaimaa
Kesto: 30 marrask. 20202 jouluk. 2020
Konferenssinumero: 18

Conference

ConferenceInternational Conference on Advances in Mobile Computing & Multimedia
LyhennettäMoMM
Maa/AlueThaimaa
KaupunkiChiang Mai
Ajanjakso30/11/202002/12/2020

Sormenjälki

Sukella tutkimusaiheisiin 'Functional Gaze Prediction in Egocentric Video'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä