Semi-parametric Approach to Random Forests for High-Dimensional Bayesian Optimisation

Vladimir Kuzmanovski*, Jaakko Hollmén

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Calibration of simulation models and hyperparameter optimisation of machine learning and deep learning methods are computationally demanding optimisation problems, for which many state-of-the-art optimisation methods are adopted and applied in various studies. However, their performances come to a test when the parameter optimisation problems exhibit high-dimensional spaces and expensive evaluation of models’ or methods’ settings. Population-based (evolutionary) methods work well for the former but not suitable for expensive evaluation functions. On the opposite, Bayesian optimisation eliminates the necessity of frequent simulations to find the global optima. However, the computational demand rises significantly as the number of parameters increases. Bayesian optimisation with random forests has overcome issues of its state-of-the-art counterparts. Still, due to the non-parametric output, it fails to utilise the capabilities of available acquisition functions. We propose a semi-parametric approach to overcome such limitations to random forests by identifying a mixture of parametric components in their outcomes. The proposed approach is evaluated empirically on four optimisation benchmark functions with varying dimensionality, confirming the improvement in guiding the search process. Finally, in terms of running time, it scales linearly with respect to the dimensionality of the search space.

Original languageEnglish
Title of host publicationDiscovery Science - 25th International Conference, DS 2022, Proceedings
EditorsPoncelet Pascal, Dino Ienco
Number of pages11
Publication statusPublished - 2022
MoE publication typeA4 Article in a conference publication
EventInternational Conference on Discovery Science - Montpellier, France
Duration: 10 Oct 202212 Oct 2022
Conference number: 25

Publication series

NameLecture Notes in Computer Science
Volume13601 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Conference on Discovery Science
Abbreviated titleDS


Dive into the research topics of 'Semi-parametric Approach to Random Forests for High-Dimensional Bayesian Optimisation'. Together they form a unique fingerprint.

Cite this