Abstract
We had previously introduced Learn + + , inspired in part by the ensemble based AdaBoost algorithm, for incrementally learning from new data, including new concept classes, without forgetting what had been previously learned. In this effort, we compare the incremental learning performance of Learn + + and AdaBoost under several combination schemes, including their native, weighted majority voting. We show on several databases that changing AdaBoost’s distribution update rule from hypothesis based update to ensemble based update allows significantly more efficient incremental learning ability, regardless of the combination rule used to combine the classifiers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
French, R.: Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences 3(4), 128–135 (1999)
Grossberg, S.: Nonlinear neural networks: principles, mechanisms and architectures. Neural Networks 1(1), 17–61 (1988)
Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental learning of analog multidimen-sional maps. IEEE Trans. on Neural Networks 3(5), 698–713 (1992)
Freund, Y., Schapire, R.: Decision-theoretic generalization of on-line learning and an application to boosting. J. Comp. Sys. Sci. 55(1), 119–139 (1997)
Polikar, R., Udpa, L., Udpa, S., Honavar, V.: Learn + + : An incremental learning algorithm for supervised neural networks. IEEE Trans. on System, Man and Cybernetics (C) 31(4), 497–508 (2001)
Polikar, R., Byorick, J., Krause, S., Marino, A., Moreton, M.: Learn++: A classifier independent incremental learning algorithm for supervised neural networks. In: Proc. of Int. Joint Conference on Neural Networks (IJCNN 2002), May 12-17, 2002, vol. 2, pp. 1742–1747. Honolulu, HI (2002)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. on Pattern Analy. and Machine Int. 20(3), 226–239 (1998)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. John Wiley & Sons, N.J (2004)
Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Rec. 34(2), 299–314 (2001)
Blake, C.L., Merz, C.J.: Univ. of California, Irvine, Repository of Machine Learning Databases at Irvine, CA
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mohammed, H.S., Leander, J., Marbach, M., Polikar, R. (2006). Can AdaBoost.M1 Learn Incrementally? A Comparison to Learn + + Under Different Combination Rules. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840817_27
Download citation
DOI: https://doi.org/10.1007/11840817_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38625-4
Online ISBN: 978-3-540-38627-8
eBook Packages: Computer ScienceComputer Science (R0)