Building heterogeneous ensembles by pooling homogeneous ensembles

Maryam Sabzevari*, Gonzalo Martínez-Muñoz, Alberto Suárez

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Heterogeneous ensembles consist of predictors of different types, which are likely to have different biases. If these biases are complementary, the combination of their decisions is beneficial and could be superior to homogeneous ensembles. In this paper, a family of heterogeneous ensembles is built by pooling classifiers from M homogeneous ensembles of different types of size T. Depending on the fraction of base classifiers of each type, a particular heterogeneous combination in this family is represented by a point in a regular simplex in M dimensions. The M vertices of this simplex represent the different homogeneous ensembles. A displacement away from one of these vertices effects a smooth transformation of the corresponding homogeneous ensemble into a heterogeneous one. The optimal composition of such heterogeneous ensemble can be determined using cross-validation or, if bootstrap samples are used to build the individual classifiers, out-of-bag data. The proposed heterogeneous ensemble building strategy, composed of neural networks, SVMs, and random trees (i.e. from a standard random forest), is analyzed in a comprehensive empirical analysis and compared to a benchmark of other heterogeneous and homogeneous ensembles. The achieved results illustrate the gains that can be achieved by the proposed ensemble creation method with respect to both homogeneous ensembles and to the tested heterogeneous building strategy at a fraction of the training cost.

Original languageEnglish
Number of pages8
JournalInternational Journal of Machine Learning and Cybernetics
DOIs
Publication statusE-pub ahead of print - 2021
MoE publication typeA1 Journal article-refereed

Keywords

  • Ensembles
  • Heterogeneous
  • Homogeneous
  • Optimal composition
  • Simplex

Fingerprint

Dive into the research topics of 'Building heterogeneous ensembles by pooling homogeneous ensembles'. Together they form a unique fingerprint.

Cite this