Abstract
The purpose of ensemble pruning is to reduce the number of predictive models in order to improve efficiency and predictive performance of the ensemble. In clustering-based approach, we are looking for groups of similar models, and then we prune each of them separately in order to increase overall diversity of the ensemble. In this paper we propose two methods for this purpose using classifier clustering on the basis of a criterion based on diversity measure. In the first method we select from each cluster the model with the best predictive performance to form the final ensemble, while the second one employs the multistage organization, where instead of removing the classifiers from the ensemble each classifier group makes the decision independently. The final answer of the proposed framework is the result of the majority voting of the decisions returned by each group. Experimentation results validated through statistical tests confirmed the usefulness of the proposed approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bakker, B., Heskes, T.: Clustering ensembles of neural network models. Neural Netw. 16(2), 261–269 (2003)
Fu, Q., Hu, S.X., Zhao, S.: Clustering-based selective neural network ensemble. J. Zhejiang Univ. Sci. 6(5), 387–392 (2005)
Giacinto, G., Roli, F., Fumera, G.: Design of effective multiple classifier systems by clustering of classifiers. In: 15th International Conference on Pattern Recognition, ICPR 2000 (2000)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 832–844 (1998)
Ho, T.K., Hull, J.J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 16(1), 66–75 (1994)
Kohavi, R., Wolpert, D.: Bias plus variance decomposition for zero-one loss functions. In: Proceedings of the Thirteenth International Conference on International Conference on Machine Learning, ICML 1996, pp. 275–283. Morgan Kaufmann Publishers Inc., San Francisco (1996)
Krawczyk, B., Cyganek, B.: Selecting locally specialised classifiers for one-class classification ensembles. Pattern Anal. Appl. 20(2), 427–439 (2017)
Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Wozniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Hoboken (2004)
Lazarevic, A., Obradovic, Z.: The effective pruning of neural network classifiers. In: 2001 IEEE/INNS International Conference on Neural Networks, IJCNN 2001 (2001)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Ruta, D., Gabrys, B.: A theoretical analysis of the limits of majority voting errors for multiple classifier systems. Pattern Anal. Appl. 2(4), 333–350 (2002)
Topolski, M.: Algorithm of principal component analysis PCA with fuzzy observation of facility features detection of carcinoma cells multiple myeloma. In: Burduk, R., Kurzynski, M., Wozniak, M. (eds.) Progress in Computer Recognition Systems (2019)
Woźniak, M., Graña, M.: A survey of multiple classifier systems as hybrid systems. Inf. Fusion 16, 3–17 (2014)
Zhang, H., Cao, L.: A spectral clustering based ensemble pruning approach. Neurocomputing 139, 289–297 (2014)
Zhou, Z.H.: Ensemble Methods: Foundations and Algorithms. Chapman & Hall CRC, Boca Raton (2012)
Acknowledgement
This work was supported by the Polish National Science Centre under the grant No. 2017/27/B/ST6/01325 as well as by the statutory funds of the Department of Systems and Computer Networks, Faculty of Electronics, Wroclaw University of Science and Technology.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zyblewski, P., Woźniak, M. (2019). Clustering-Based Ensemble Pruning and Multistage Organization Using Diversity. In: Pérez García, H., Sánchez González, L., Castejón Limas, M., Quintián Pardo, H., Corchado Rodríguez, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2019. Lecture Notes in Computer Science(), vol 11734. Springer, Cham. https://doi.org/10.1007/978-3-030-29859-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-29859-3_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29858-6
Online ISBN: 978-3-030-29859-3
eBook Packages: Computer ScienceComputer Science (R0)