Gyroscope-aided motion deblurring with deep networks

Janne Mustaniemi; Juho Kannala; Simo Särkkä; Jiri Matas; Janne Heikkilä

doi:10.1109/WACV.2019.00208

Gyroscope-aided motion deblurring with deep networks

Janne Mustaniemi^*, Juho Kannala, Simo Särkkä, Jiri Matas, Janne Heikkilä

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

32 Citations (Scopus)

Abstract

We propose a deblurring method that incorporates gyroscope measurements into a convolutional neural network (CNN). With the help of such measurements, it can handle extremely strong and spatially-variant motion blur. At the same time, the image data is used to overcome the limitations of gyro-based blur estimation. To train our network, we also introduce a novel way of generating realistic training data using the gyroscope. The evaluation shows a clear improvement in visual quality over the state-of-the-art while achieving real-time performance. Furthermore, the method is shown to improve the performance of existing feature detectors and descriptors against the motion blur.

Original language	English
Title of host publication	Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019
Publisher	IEEE
Pages	1914-1922
Number of pages	9
ISBN (Electronic)	9781728119755
DOIs	https://doi.org/10.1109/WACV.2019.00208
Publication status	Published - 4 Mar 2019
MoE publication type	A4 Conference publication
Event	IEEE Winter Conference on Applications of Computer Vision - Waikoloa Village, United States Duration: 7 Jan 2019 → 11 Jan 2019 Conference number: 19

Publication series

Name	IEEE Winter Conference on Applications of Computer Vision
Publisher	IEEE
ISSN (Print)	2472-6737

Conference

Conference	IEEE Winter Conference on Applications of Computer Vision
Abbreviated title	WACV
Country/Territory	United States
City	Waikoloa Village
Period	07/01/2019 → 11/01/2019

Keywords

Cameras
Gyroscopes
Image motion analysis
Image restoration

Access to Document

10.1109/WACV.2019.00208

https://arxiv.org/abs/1810.00986v2

Cite this

@inproceedings{aaf6d97d4a7e45a19e9b742f72f05937,

title = "Gyroscope-aided motion deblurring with deep networks",

abstract = "We propose a deblurring method that incorporates gyroscope measurements into a convolutional neural network (CNN). With the help of such measurements, it can handle extremely strong and spatially-variant motion blur. At the same time, the image data is used to overcome the limitations of gyro-based blur estimation. To train our network, we also introduce a novel way of generating realistic training data using the gyroscope. The evaluation shows a clear improvement in visual quality over the state-of-the-art while achieving real-time performance. Furthermore, the method is shown to improve the performance of existing feature detectors and descriptors against the motion blur.",

keywords = "Cameras, Gyroscopes, Image motion analysis, Image restoration",

author = "Janne Mustaniemi and Juho Kannala and Simo S{\"a}rkk{\"a} and Jiri Matas and Janne Heikkil{\"a}",

year = "2019",

month = mar,

day = "4",

doi = "10.1109/WACV.2019.00208",

language = "English",

series = "IEEE Winter Conference on Applications of Computer Vision",

publisher = "IEEE",

pages = "1914--1922",

booktitle = "Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",

address = "United States",

note = "IEEE Winter Conference on Applications of Computer Vision, WACV ; Conference date: 07-01-2019 Through 11-01-2019",

}

Mustaniemi, J, Kannala, J , Särkkä, S, Matas, J & Heikkilä, J 2019, Gyroscope-aided motion deblurring with deep networks. in Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019., 8658406, IEEE Winter Conference on Applications of Computer Vision, IEEE, pp. 1914-1922, IEEE Winter Conference on Applications of Computer Vision, Waikoloa Village, Hawaii, United States, 07/01/2019. https://doi.org/10.1109/WACV.2019.00208

Gyroscope-aided motion deblurring with deep networks. / Mustaniemi, Janne; Kannala, Juho ; Särkkä, Simo et al.
Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. IEEE, 2019. p. 1914-1922 8658406 (IEEE Winter Conference on Applications of Computer Vision).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Gyroscope-aided motion deblurring with deep networks

AU - Mustaniemi, Janne

AU - Kannala, Juho

AU - Särkkä, Simo

AU - Matas, Jiri

AU - Heikkilä, Janne

N1 - Conference code: 19

PY - 2019/3/4

Y1 - 2019/3/4

N2 - We propose a deblurring method that incorporates gyroscope measurements into a convolutional neural network (CNN). With the help of such measurements, it can handle extremely strong and spatially-variant motion blur. At the same time, the image data is used to overcome the limitations of gyro-based blur estimation. To train our network, we also introduce a novel way of generating realistic training data using the gyroscope. The evaluation shows a clear improvement in visual quality over the state-of-the-art while achieving real-time performance. Furthermore, the method is shown to improve the performance of existing feature detectors and descriptors against the motion blur.

AB - We propose a deblurring method that incorporates gyroscope measurements into a convolutional neural network (CNN). With the help of such measurements, it can handle extremely strong and spatially-variant motion blur. At the same time, the image data is used to overcome the limitations of gyro-based blur estimation. To train our network, we also introduce a novel way of generating realistic training data using the gyroscope. The evaluation shows a clear improvement in visual quality over the state-of-the-art while achieving real-time performance. Furthermore, the method is shown to improve the performance of existing feature detectors and descriptors against the motion blur.

KW - Cameras

KW - Gyroscopes

KW - Image motion analysis

KW - Image restoration

UR - http://www.scopus.com/inward/record.url?scp=85063595270&partnerID=8YFLogxK

U2 - 10.1109/WACV.2019.00208

DO - 10.1109/WACV.2019.00208

M3 - Conference article in proceedings

AN - SCOPUS:85063595270

T3 - IEEE Winter Conference on Applications of Computer Vision

SP - 1914

EP - 1922

BT - Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

PB - IEEE

T2 - IEEE Winter Conference on Applications of Computer Vision

Y2 - 7 January 2019 through 11 January 2019

ER -

Gyroscope-aided motion deblurring with deep networks

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this