Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion

Otto Seiskari*, Jerry Ylilammi, Valtteri Kaatrasalo, Pekka Rantalankila, Matias Turkulainen, Juho Kannala, Esa Rahtu, Arno Solin

*Tämän työn vastaava kirjoittaja

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

Abstrakti

High-quality scene reconstruction and novel view synthesis based on Gaussian Splatting (3DGS) typically require steady, high-quality photographs, often impractical to capture with handheld cameras. We present a method that adapts to camera motion and allows high-quality scene reconstruction with handheld video data suffering from motion blur and rolling shutter distortion. Our approach is based on detailed modelling of the physical image formation process and utilizes velocities estimated using visual-inertial odometry (VIO). Camera poses are considered non-static during the exposure time of a single image frame and camera poses are further optimized in the reconstruction process. We formulate a differentiable rendering pipeline that leverages screen space approximation to efficiently incorporate rolling-shutter and motion blur effects into the 3DGS framework. Our results with both synthetic and real data demonstrate superior performance in mitigating camera motion over existing methods, thereby advancing 3DGS in naturalistic settings.
AlkuperäiskieliEnglanti
OtsikkoComputer Vision – ECCV 2024
Alaotsikko18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LXXI
ToimittajatAleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol
KustantajaSpringer
Sivut160-177
ISBN (elektroninen)978-3-031-73209-6
ISBN (painettu)978-3-031-73208-9
DOI - pysyväislinkit
TilaJulkaistu - 1 marrask. 2024
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaEuropean Conference on Computer Vision - Milano, Italia
Kesto: 29 syysk. 20244 lokak. 2024
Konferenssinumero: 18

Julkaisusarja

NimiLecture Notes in Computer Science
KustantajaSpringer
Vuosikerta15129
ISSN (painettu)0302-9743
ISSN (elektroninen)1611-3349

Conference

ConferenceEuropean Conference on Computer Vision
LyhennettäECCV
Maa/AlueItalia
KaupunkiMilano
Ajanjakso29/09/202404/10/2024

Sormenjälki

Sukella tutkimusaiheisiin 'Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä