Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution

Topi Paananen; Juho Piironen; Michael Andersen; Aki Vehtari

Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution

Topi Paananen
, Juho Piironen
, Michael Andersen
, Aki Vehtari

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

182 Downloads (Pure)

Abstract

Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse lengthscale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input
variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results
on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.

Original language	English
Title of host publication	Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics
Publisher	JMLR
Number of pages	10
Publication status	Published - 16 Apr 2019
MoE publication type	A4 Conference publication
Event	International Conference on Artificial Intelligence and Statistics - Naha, Japan Duration: 16 Apr 2019 → 18 Apr 2019 Conference number: 22

Publication series

Name	Proceedings of Machine Learning Research
Publisher	PMLR
Volume	89
ISSN (Print)	1938-7228

Conference

Conference	International Conference on Artificial Intelligence and Statistics
Abbreviated title	AISTATS
Country/Territory	Japan
City	Naha
Period	16/04/2019 → 18/04/2019

Access to Document

paananen19aFinal published version, 2.96 MB

http://proceedings.mlr.press/v89/paananen19a.html

Cite this

Paananen, T., Piironen, J., Andersen, M., & Vehtari, A. (2019). Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research; Vol. 89). JMLR. http://proceedings.mlr.press/v89/paananen19a.html

@inproceedings{b1e288949b2142af842847b6293a277f,

title = "Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution",

abstract = "Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse lengthscale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.",

author = "Topi Paananen and Juho Piironen and Michael Andersen and Aki Vehtari",

year = "2019",

month = apr,

day = "16",

language = "English",

series = "Proceedings of Machine Learning Research",

publisher = "JMLR",

booktitle = "Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics",

address = "United States",

note = "International Conference on Artificial Intelligence and Statistics, AISTATS ; Conference date: 16-04-2019 Through 18-04-2019",

}

Paananen, T, Piironen, J, Andersen, M & Vehtari, A 2019, Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution. in Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 89, JMLR, International Conference on Artificial Intelligence and Statistics, Naha, Japan, 16/04/2019. <http://proceedings.mlr.press/v89/paananen19a.html>

Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution. / Paananen, Topi; Piironen, Juho; Andersen, Michael et al.
Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics. JMLR, 2019. (Proceedings of Machine Learning Research; Vol. 89).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution

AU - Paananen, Topi

AU - Piironen, Juho

AU - Andersen, Michael

AU - Vehtari, Aki

N1 - Conference code: 22

PY - 2019/4/16

Y1 - 2019/4/16

N2 - Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse lengthscale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.

AB - Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse lengthscale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables in terms of predictive performance. To improve on this, we propose two novel variable selection methods for Gaussian process models that utilize the predictions of a full model in the vicinity of the training points and thereby rank the variables based on their predictive relevance. Our empirical results on synthetic and real world data sets demonstrate improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.

UR - https://arxiv.org/abs/1712.08048

UR - https://github.com/topipa/gp-varsel-kl-var

M3 - Conference article in proceedings

T3 - Proceedings of Machine Learning Research

BT - Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics

PB - JMLR

T2 - International Conference on Artificial Intelligence and Statistics

Y2 - 16 April 2019 through 18 April 2019

ER -

Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this