Abstract
Since extreme learning machine (ELM) was proposed, it has been found that some hidden nodes in ELM may play a very minor role in the network output. To avoid this problem, enhanced random search based incremental extreme learning machine (EI-ELM) is proposed. However, we find that the EI-ELM’s training time is too long. In addition, EI-ELM can only add hidden nodes one by one. This paper proposes a fast method for EI-ELM (referred to as FI-ELM). At each learning step, several hidden nodes are randomly generated and the hidden nodes selected by the multiresponse sparse regression (MRSR) are added to the existing network. The output weights of the network are updated by a fast iterative method. The experimental results show that compared with EI-ELM, FI-ELM spends less time on training. Taking this advantage, FI-ELM can generate more hidden nodes to find the hidden node leading to larger residual error decreasing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhou, G., Zhao, Q., Zhang, Y., Adalı, T., Xie, S., Cichocki, A.: Linked component analysis from matrices to high-order tensors: applications to biomedical data. Proc. IEEE 104(2), 310–331 (2016)
Zhou, G., Cichocki, A., Zhang, Y., Mandic, D.P.: Group component analysis for multiblock data: common and individual feature extraction. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2426–2439 (2016)
Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)
Lan, Y., Soh, Y., Huang, G.B.: Ensemble of online sequential extreme learning machine. Neurocomputing 72(13–15), 3391–3395 (2009)
Lan, Y., Soh, Y., Huang, G.B.: Random search enhancement of error minimized extreme learning machine. In: European Symposium on Artificial Neural Networks, Esann 2010, Bruges (2010)
Huang, G.B., Chen, L.: Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18), 3460–3468 (2008)
Zhang, R., Lan, Y., Huang, G.B., Xu, Z.B., Soh, Y.C.: Dynamic extreme learning machine and its approximation capability. IEEE Trans. Cybern. 43(6), 2054–2065 (2013)
Huang, G.B., Chen, L.: Convex incremental extreme learning machine. Neurocomputing 70(16–18), 3056–3062 (2007)
Rong, H.J., Ong, Y.S., Tan, A.H., Zhu, Z.X.: A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1–3), 359–366 (2008)
Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. B 42(2), 513–529 (2012)
Wan, Y., Song, S., Huang, G.: Incremental extreme learning machine based on cascade neural networks. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1889–1894. IEEE (2015)
Similä, T., Tikka, J.: Multiresponse sparse regression with application to multidimensional scaling. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 97–102. Springer, Heidelberg (2005). doi:10.1007/11550907_16
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)
Zhang, F.: The Schur Complement and Its Applications. Numerical Methods and Algorithms, vol. 4. Springer, Boston (2005)
UCI repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html
Acknowledgments
The work is supported by National Natural Science Foundation of China (61372142, U1401252), Fundamental Research Funds for the Central Universities SCUT (2017MS062), Guangzhou city science and technology research projects(201508010023, 201604016133).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lao, Z., Zhou, Z., Huang, J. (2017). Incremental Extreme Learning Machine via Fast Random Search Method. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10634. Springer, Cham. https://doi.org/10.1007/978-3-319-70087-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-70087-8_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70086-1
Online ISBN: 978-3-319-70087-8
eBook Packages: Computer ScienceComputer Science (R0)