A connection between extreme learning machine and neural network kernel

Eli Parviainen; Jaakko Riihimäki

doi:10.1007/978-3-642-29764-9-8

A connection between extreme learning machine and neural network kernel

Eli Parviainen^*
, Jaakko Riihimäki

^*Corresponding author for this work

Department of Neuroscience and Biomedical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

11 Citations (Scopus)

Abstract

We study a connection between extreme learning machine (ELM) and neural network kernel (NNK). NNK is derived from a neural network with an infinite number of hidden units. We interpret ELM as an approximation to this infinite network. We show that ELM and NNK can, to certain extent, replace each other. ELM can be used to form a kernel, and NNK can be decomposed into feature vectors to be used in the hidden layer of ELM. The connection reveals possible importance of weight variance as a parameter of ELM. Based on our experiments, we recommend that model selection on ELM should consider not only the number of hidden units, as is the current practice, but also the variance of weights. We also study the interaction of variance and the number of hidden units, and discuss some properties of ELM, that may have been too strongly interpreted previously.

Original language	English
Title of host publication	Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers
Publisher	Springer
Pages	122-135
Number of pages	14
Volume	272 CCIS
ISBN (Print)	9783642297632
DOIs	https://doi.org/10.1007/978-3-642-29764-9-8
Publication status	Published - 2013
MoE publication type	A4 Conference publication
Event	International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Valencia, Spain Duration: 25 Oct 2010 → 28 Oct 2010 Conference number: 2

Publication series

Name	Communications in Computer and Information Science
Volume	272 CCIS
ISSN (Print)	1865-0929

Conference

Conference	International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management
Abbreviated title	IC3K
Country/Territory	Spain
City	Valencia
Period	25/10/2010 → 28/10/2010

Keywords

ELM
Extreme learning machine
Neural network kernel

Access to Document

10.1007/978-3-642-29764-9-8

Cite this

Parviainen, E., & Riihimäki, J. (2013). A connection between extreme learning machine and neural network kernel. In Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers (Vol. 272 CCIS, pp. 122-135). (Communications in Computer and Information Science; Vol. 272 CCIS). Springer. https://doi.org/10.1007/978-3-642-29764-9-8

@inproceedings{6ded69b3b4a44d5cab212c96300dc3f7,

title = "A connection between extreme learning machine and neural network kernel",

abstract = "We study a connection between extreme learning machine (ELM) and neural network kernel (NNK). NNK is derived from a neural network with an infinite number of hidden units. We interpret ELM as an approximation to this infinite network. We show that ELM and NNK can, to certain extent, replace each other. ELM can be used to form a kernel, and NNK can be decomposed into feature vectors to be used in the hidden layer of ELM. The connection reveals possible importance of weight variance as a parameter of ELM. Based on our experiments, we recommend that model selection on ELM should consider not only the number of hidden units, as is the current practice, but also the variance of weights. We also study the interaction of variance and the number of hidden units, and discuss some properties of ELM, that may have been too strongly interpreted previously.",

keywords = "ELM, Extreme learning machine, Neural network kernel",

author = "Eli Parviainen and Jaakko Riihim{\"a}ki",

year = "2013",

doi = "10.1007/978-3-642-29764-9-8",

language = "English",

isbn = "9783642297632",

volume = "272 CCIS",

series = "Communications in Computer and Information Science",

publisher = "Springer",

pages = "122--135",

booktitle = "Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers",

address = "Germany",

note = "International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, IC3K ; Conference date: 25-10-2010 Through 28-10-2010",

}

Parviainen, E & Riihimäki, J 2013, A connection between extreme learning machine and neural network kernel. in Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers. vol. 272 CCIS, Communications in Computer and Information Science, vol. 272 CCIS, Springer, pp. 122-135, International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, Valencia, Spain, 25/10/2010. https://doi.org/10.1007/978-3-642-29764-9-8

A connection between extreme learning machine and neural network kernel. / Parviainen, Eli; Riihimäki, Jaakko.
Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers. Vol. 272 CCIS Springer, 2013. p. 122-135 (Communications in Computer and Information Science; Vol. 272 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - A connection between extreme learning machine and neural network kernel

AU - Parviainen, Eli

AU - Riihimäki, Jaakko

N1 - Conference code: 2

PY - 2013

Y1 - 2013

N2 - We study a connection between extreme learning machine (ELM) and neural network kernel (NNK). NNK is derived from a neural network with an infinite number of hidden units. We interpret ELM as an approximation to this infinite network. We show that ELM and NNK can, to certain extent, replace each other. ELM can be used to form a kernel, and NNK can be decomposed into feature vectors to be used in the hidden layer of ELM. The connection reveals possible importance of weight variance as a parameter of ELM. Based on our experiments, we recommend that model selection on ELM should consider not only the number of hidden units, as is the current practice, but also the variance of weights. We also study the interaction of variance and the number of hidden units, and discuss some properties of ELM, that may have been too strongly interpreted previously.

AB - We study a connection between extreme learning machine (ELM) and neural network kernel (NNK). NNK is derived from a neural network with an infinite number of hidden units. We interpret ELM as an approximation to this infinite network. We show that ELM and NNK can, to certain extent, replace each other. ELM can be used to form a kernel, and NNK can be decomposed into feature vectors to be used in the hidden layer of ELM. The connection reveals possible importance of weight variance as a parameter of ELM. Based on our experiments, we recommend that model selection on ELM should consider not only the number of hidden units, as is the current practice, but also the variance of weights. We also study the interaction of variance and the number of hidden units, and discuss some properties of ELM, that may have been too strongly interpreted previously.

KW - ELM

KW - Extreme learning machine

KW - Neural network kernel

UR - https://www.scopus.com/pages/publications/84873922670

U2 - 10.1007/978-3-642-29764-9-8

DO - 10.1007/978-3-642-29764-9-8

M3 - Conference article in proceedings

AN - SCOPUS:84873922670

SN - 9783642297632

VL - 272 CCIS

T3 - Communications in Computer and Information Science

SP - 122

EP - 135

BT - Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers

PB - Springer

T2 - International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management

Y2 - 25 October 2010 through 28 October 2010

ER -

Parviainen E, Riihimäki J. A connection between extreme learning machine and neural network kernel. In Knowledge Discovery, Knowledge Engineering and Knowledge Management - Second International Joint Conference, IC3K 2010, Revised Selected Papers. Vol. 272 CCIS. Springer. 2013. p. 122-135. (Communications in Computer and Information Science). doi: 10.1007/978-3-642-29764-9-8

A connection between extreme learning machine and neural network kernel

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this