Sample-based Monte Carlo Denoising using a Kernel-Splatting Network

Research output: Contribution to journalArticle

Standard

Sample-based Monte Carlo Denoising using a Kernel-Splatting Network. / Gharbi, Michaël; Li, Tzu-Mao; Aittala, Miika; Lehtinen, Jaakko; Durand, Frédo.

In: ACM Transactions on Graphics, Vol. 38, No. 4, 125, 07.2019.

Research output: Contribution to journalArticle

Harvard

APA

Vancouver

Author

Bibtex - Download

@article{2328dd1caedb48159dfed710d54b9c3c,
title = "Sample-based Monte Carlo Denoising using a Kernel-Splatting Network",
abstract = "Denoising has proven to be useful to efficiently generate high-quality Monte Carlo renderings. Traditional pixel-based denoisers exploit summary statistics of a pixel's sample distributions, which discards much of the samples' information and limits their denoising power. On the other hand, sample-based techniques tend to be slow and have difficulties handling general transport scenarios. We present the first convolutional network that can learn to denoise Monte Carlo renderings directly from the samples. Learning the mapping between samples and images creates new challenges for the network architecture design: the order of the samples is arbitrary, and they should be treated in a permutation invariant manner. To address these challenges, we develop a novel kernel-predicting architecture that splats individual samples onto nearby pixels. Splatting is a natural solution to situations such as motion blur, depth-of-field and many light transport paths, where it is easier to predict which pixels a sample contributes to, rather than a gather approach that needs to figure out, for each pixel, which samples (or nearby pixels) are relevant. Compared to previous state-of-the-art methods, ours is robust to the severe noise of low-sample count images (e.g. 8 samples per pixel) and yields higher-quality results both visually and numerically. Our approach retains the generality and efficiency of pixel-space methods while enjoying the expressiveness and accuracy of the more complex sample-based approaches.",
keywords = "Monte Carlo denoising, deep learning, data-driven methods, convolutional neural networks",
author = "Micha{\"e}l Gharbi and Tzu-Mao Li and Miika Aittala and Jaakko Lehtinen and Fr{\'e}do Durand",
year = "2019",
month = "7",
doi = "10.1145/3306346.3322954",
language = "English",
volume = "38",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
number = "4",

}

RIS - Download

TY - JOUR

T1 - Sample-based Monte Carlo Denoising using a Kernel-Splatting Network

AU - Gharbi, Michaël

AU - Li, Tzu-Mao

AU - Aittala, Miika

AU - Lehtinen, Jaakko

AU - Durand, Frédo

PY - 2019/7

Y1 - 2019/7

N2 - Denoising has proven to be useful to efficiently generate high-quality Monte Carlo renderings. Traditional pixel-based denoisers exploit summary statistics of a pixel's sample distributions, which discards much of the samples' information and limits their denoising power. On the other hand, sample-based techniques tend to be slow and have difficulties handling general transport scenarios. We present the first convolutional network that can learn to denoise Monte Carlo renderings directly from the samples. Learning the mapping between samples and images creates new challenges for the network architecture design: the order of the samples is arbitrary, and they should be treated in a permutation invariant manner. To address these challenges, we develop a novel kernel-predicting architecture that splats individual samples onto nearby pixels. Splatting is a natural solution to situations such as motion blur, depth-of-field and many light transport paths, where it is easier to predict which pixels a sample contributes to, rather than a gather approach that needs to figure out, for each pixel, which samples (or nearby pixels) are relevant. Compared to previous state-of-the-art methods, ours is robust to the severe noise of low-sample count images (e.g. 8 samples per pixel) and yields higher-quality results both visually and numerically. Our approach retains the generality and efficiency of pixel-space methods while enjoying the expressiveness and accuracy of the more complex sample-based approaches.

AB - Denoising has proven to be useful to efficiently generate high-quality Monte Carlo renderings. Traditional pixel-based denoisers exploit summary statistics of a pixel's sample distributions, which discards much of the samples' information and limits their denoising power. On the other hand, sample-based techniques tend to be slow and have difficulties handling general transport scenarios. We present the first convolutional network that can learn to denoise Monte Carlo renderings directly from the samples. Learning the mapping between samples and images creates new challenges for the network architecture design: the order of the samples is arbitrary, and they should be treated in a permutation invariant manner. To address these challenges, we develop a novel kernel-predicting architecture that splats individual samples onto nearby pixels. Splatting is a natural solution to situations such as motion blur, depth-of-field and many light transport paths, where it is easier to predict which pixels a sample contributes to, rather than a gather approach that needs to figure out, for each pixel, which samples (or nearby pixels) are relevant. Compared to previous state-of-the-art methods, ours is robust to the severe noise of low-sample count images (e.g. 8 samples per pixel) and yields higher-quality results both visually and numerically. Our approach retains the generality and efficiency of pixel-space methods while enjoying the expressiveness and accuracy of the more complex sample-based approaches.

KW - Monte Carlo denoising

KW - deep learning

KW - data-driven methods

KW - convolutional neural networks

U2 - 10.1145/3306346.3322954

DO - 10.1145/3306346.3322954

M3 - Article

VL - 38

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 4

M1 - 125

ER -

ID: 36041740