Expansion of Visual Hints for Improved Generalization in Stereo Matching

Andrea Pilzer; Yuxin Hou; Niki Loppi; Arno Solin; Juho Kannala

doi:10.1109/WACV56688.2023.00579

Expansion of Visual Hints for Improved Generalization in Stereo Matching

Andrea Pilzer^*, Yuxin Hou, Niki Loppi, Arno Solin, Juho Kannala

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

3 Citations (Scopus)

Abstract

We introduce visual hints expansion for guiding stereo matching to improve generalization. Our work is motivated by the robustness of Visual Inertial Odometry (VIO) in computer vision and robotics, where a sparse and unevenly distributed set of feature points characterizes a scene. To improve stereo matching, we propose to elevate 2D hints to 3D points. These sparse and unevenly distributed 3D visual hints are expanded using a 3D random geometric graph, which enhances the learning and inference process. We evaluate our proposal on multiple widely adopted benchmarks and show improved performance without access to additional sensors other than the image sequence. To highlight practical applicability and symbiosis with visual odometry, we demonstrate how our methods run on embedded hardware.

Original language	English
Title of host publication	Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023
Publisher	IEEE
Pages	5829-5838
Number of pages	10
ISBN (Electronic)	978-1-6654-9346-8
DOIs	https://doi.org/10.1109/WACV56688.2023.00579
Publication status	Published - 2023
MoE publication type	A4 Conference publication
Event	IEEE Winter Conference on Applications of Computer Vision - Waikoloa, United States Duration: 2 Jan 2023 → 7 Jan 2023

Publication series

Name	IEEE Winter Conference on Applications of Computer Vision
ISSN (Electronic)	2642-9381

Conference

Conference	IEEE Winter Conference on Applications of Computer Vision
Abbreviated title	WACV
Country/Territory	United States
City	Waikoloa
Period	02/01/2023 → 07/01/2023

Keywords

Algorithms: 3D computer vision

Access to Document

10.1109/WACV56688.2023.00579

https://arxiv.org/abs/2211.00392

Cite this

@inproceedings{987886abb85f4420ae4ea97b6e780755,

title = "Expansion of Visual Hints for Improved Generalization in Stereo Matching",

abstract = "We introduce visual hints expansion for guiding stereo matching to improve generalization. Our work is motivated by the robustness of Visual Inertial Odometry (VIO) in computer vision and robotics, where a sparse and unevenly distributed set of feature points characterizes a scene. To improve stereo matching, we propose to elevate 2D hints to 3D points. These sparse and unevenly distributed 3D visual hints are expanded using a 3D random geometric graph, which enhances the learning and inference process. We evaluate our proposal on multiple widely adopted benchmarks and show improved performance without access to additional sensors other than the image sequence. To highlight practical applicability and symbiosis with visual odometry, we demonstrate how our methods run on embedded hardware.",

keywords = "Algorithms: 3D computer vision",

author = "Andrea Pilzer and Yuxin Hou and Niki Loppi and Arno Solin and Juho Kannala",

note = "Funding Information: Acknowledgements. We acknowledge funding from Academy of Finland (No. 339730, 324345), and the Finnish Center for Artificial Intelligence (FCAI). We acknowledge the computational resources by the Aalto Science-IT project and CSC – IT Center for Science, Finland. Publisher Copyright: {\textcopyright} 2023 IEEE.; IEEE Winter Conference on Applications of Computer Vision, WACV ; Conference date: 02-01-2023 Through 07-01-2023",

year = "2023",

doi = "10.1109/WACV56688.2023.00579",

language = "English",

series = "IEEE Winter Conference on Applications of Computer Vision",

publisher = "IEEE",

pages = "5829--5838",

booktitle = "Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023",

address = "United States",

}

Pilzer, A, Hou, Y, Loppi, N, Solin, A & Kannala, J 2023, Expansion of Visual Hints for Improved Generalization in Stereo Matching. in Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023. IEEE Winter Conference on Applications of Computer Vision, IEEE, pp. 5829-5838, IEEE Winter Conference on Applications of Computer Vision, Waikoloa, Hawaii, United States, 02/01/2023. https://doi.org/10.1109/WACV56688.2023.00579

Expansion of Visual Hints for Improved Generalization in Stereo Matching. / Pilzer, Andrea; Hou, Yuxin; Loppi, Niki et al.
Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023. IEEE, 2023. p. 5829-5838 (IEEE Winter Conference on Applications of Computer Vision).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Expansion of Visual Hints for Improved Generalization in Stereo Matching

AU - Pilzer, Andrea

AU - Hou, Yuxin

AU - Loppi, Niki

AU - Solin, Arno

AU - Kannala, Juho

N1 - Funding Information: Acknowledgements. We acknowledge funding from Academy of Finland (No. 339730, 324345), and the Finnish Center for Artificial Intelligence (FCAI). We acknowledge the computational resources by the Aalto Science-IT project and CSC – IT Center for Science, Finland. Publisher Copyright: © 2023 IEEE.

PY - 2023

Y1 - 2023

N2 - We introduce visual hints expansion for guiding stereo matching to improve generalization. Our work is motivated by the robustness of Visual Inertial Odometry (VIO) in computer vision and robotics, where a sparse and unevenly distributed set of feature points characterizes a scene. To improve stereo matching, we propose to elevate 2D hints to 3D points. These sparse and unevenly distributed 3D visual hints are expanded using a 3D random geometric graph, which enhances the learning and inference process. We evaluate our proposal on multiple widely adopted benchmarks and show improved performance without access to additional sensors other than the image sequence. To highlight practical applicability and symbiosis with visual odometry, we demonstrate how our methods run on embedded hardware.

AB - We introduce visual hints expansion for guiding stereo matching to improve generalization. Our work is motivated by the robustness of Visual Inertial Odometry (VIO) in computer vision and robotics, where a sparse and unevenly distributed set of feature points characterizes a scene. To improve stereo matching, we propose to elevate 2D hints to 3D points. These sparse and unevenly distributed 3D visual hints are expanded using a 3D random geometric graph, which enhances the learning and inference process. We evaluate our proposal on multiple widely adopted benchmarks and show improved performance without access to additional sensors other than the image sequence. To highlight practical applicability and symbiosis with visual odometry, we demonstrate how our methods run on embedded hardware.

KW - Algorithms: 3D computer vision

UR - http://www.scopus.com/inward/record.url?scp=85149031074&partnerID=8YFLogxK

U2 - 10.1109/WACV56688.2023.00579

DO - 10.1109/WACV56688.2023.00579

M3 - Conference article in proceedings

AN - SCOPUS:85149031074

T3 - IEEE Winter Conference on Applications of Computer Vision

SP - 5829

EP - 5838

BT - Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023

PB - IEEE

T2 - IEEE Winter Conference on Applications of Computer Vision

Y2 - 2 January 2023 through 7 January 2023

ER -

Expansion of Visual Hints for Improved Generalization in Stereo Matching

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Solin Arno /AoF Fellow Salary: Probabilistic principles for latent space exploration in deep learning

-: Shallow models meet deep vision

-: Finnish Center for Artificial Intelligence

Science-IT

Cite this

Expansion of Visual Hints for Improved Generalization in Stereo Matching

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Solin Arno /AoF Fellow Salary: Probabilistic principles for latent space exploration in deep learning

-: Shallow models meet deep vision

-: Finnish Center for Artificial Intelligence

Equipment

Science-IT

Cite this