Abstract
The concept of Diversity is now recognized as a key characteristic of successful ensembles of predictors. In this paper we investigate an algorithm to generate diversity locally in regression ensembles of neural networks, which is based on the idea of imposing a neighborhood relation over the set of learners. In this algorithm each predictor iteratively improves its state considering only information about the performance of the neighbors to generate a sort of local negative correlation. We will assess our technique on two real data sets and compare this with Negative Correlation Learning, an effective technique to get diverse ensembles. We will demonstrate that the local approach exhibits better or comparable results than this global one.
This work was supported in part by Research Grant Fondecyt (Chile) 1040365 and 7050205, and in part by Research Grant DGIP-UTFSM (Chile). Partial support was also received from Research Grant BMBF (Germany) CHL 03-Z13.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vedelsby, J., Krogh, A.: Neural network ensembles, cross-validation and active learning. Neural Information Processing Systems 7, 231–238 (1995)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
Brown, G.: Diversity in neural network ensembles, Ph.D. thesis, School of Computer Science, University of Birmingham (2003)
Fahlman, S., Lebiere, C.: The cascade-correlation learning architecture. In: Advances in Neural Information Processing Systems, vol. 2, pp. 524–532. Morgan Kaufmann, San Francisco (1990)
Harris, R., Brown, G., Wyatt, J., Yao, X.: Diversity creation methods: A survey and categorisation. Information Fusion Journal (Special issue on Diversity in Multiple Classifier Systems) 6(1), 5–20 (2004)
Whitaker, C., Kuncheva, L.: Measures of diversity in classifier ensembles. Machine Learning 51, 181–207 (2003)
Kittler, J., Roli, F., Oza, N.C., Polikar, R. (eds.): MCS 2005. LNCS, vol. 3541. Springer, Heidelberg (2005)
Druovec, T.W., Povalej, P., Kokol, P., Stiglic, B.: Machine-learning with cellular automata. In: Advances in Intelligent Data Analysis VI, vol. 1, pp. 305–315. Springer, Heidelberg (2005)
Rosen, B.: Ensemble learning using decorrelated neural networks. Connection Science 8(3-4), 373–384 (1999)
Vlachos, P.: StatLib datasets archive (2005)
Yao, X., Lui, Y.: Ensemble learning via negative correlation. Neural Networks 12(10), 1399–1404 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ñanculef, R., Valle, C., Allende, H., Moraga, C. (2006). Ensemble Learning with Local Diversity. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840817_28
Download citation
DOI: https://doi.org/10.1007/11840817_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38625-4
Online ISBN: 978-3-540-38627-8
eBook Packages: Computer ScienceComputer Science (R0)