Abstract
A solution to the multi-class classification problem is proposed founded on the concept of an ensemble of classifiers arranged in a hierarchical binary tree formation. An issue with this solution is that if a miss-classification occurs early on in the process (near the start of the hierarchy) there is no possibility of rectifying this error later on in the process. To address this issue a multi-path strategy is investigated based on the idea of using Classification Association Rule Miners at individual nodes. The conjectured advantage offered is that the confidence values associated with this form of classifier can be used to inform the proposed multi-path strategy. More specifically the confidence values are used to determine, at each node, whether one or two paths should be followed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Agrawal, R., Srikant, R.: Fast algorithms for mining association rules. In: VLDB 1994, pp. 487–499 (1994)
Alshdaifat, E., Coenen, F., Dures, K.: Hierarchical classification for solving multi-class problems: A new approach using naive bayesian classification. In: Motoda, H., Wu, Z., Cao, L., Zaiane, O., Yao, M., Wang, W. (eds.) ADMA 2013, Part I. LNCS, vol. 8346, pp. 493–504. Springer, Heidelberg (2013)
Alshdaifat, E., Coenen, F., Dures, K.: Hierarchical single label classification: An alternative approach. In: Bramer, M., Petridis, M. (eds.) SGAI Conf., pp. 39–52. Springer (2013)
Athimethphat, M., Lerteerawong, B.: Binary classification tree for multiclass classification with observation-based clustering. In: 2012 9th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 1–4 (2012)
Bache, K., Lichman, M.: UCI machine learning repository (2013), http://archive.ics.uci.edu/ml
Bay, S.D.: Combining nearest neighbor classifiers through multiple feature subsets. In: Proc. 17th Intl. Conf. on Machine Learning, pp. 37–45. Morgan Kaufmann (1998)
Beygelzimer, A., Langford, J., Ravikumar, P.: Multiclass Classification with Filter Trees (June 2007), http://hunch.net/~jl/projects/reductions/mc_to_b/invertedTree.pdf
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Random forests. In: Machine Learning, pp. 5–32 (2001)
Chen, Y., Crawford, M.M., Ghosh, J.: Integrating support vector machines in a hierarchical output decomposition framework. In: 2004 International Geosci. and Remote Sens. Symposium, pp. 949–953 (2004)
Coenen, F.: The LUCS-KDD discretised/normalised arm and carm data library (2003), http://www.csc.liv.ac.uk/~frans/KDD/Software/LUCS_KDD_DN
Coenen, F., Leng, P.: The effect of threshold values on association rule based classification accuracy. Journal of Data and Knowledge Engineering 60(2), 345–360 (2007)
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000), http://dl.acm.org/citation.cfm?id=648054.743935
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. JAIR (1995)
Freund, Y., Schapire, R., Abe, N.: A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5), 771–780 (1999)
Han, J., Pei, J., Yin, Y.: Mining frequent patterns without candidate generation. SIGMOD Rec. 29(2), 1–12 (2000), http://doi.acm.org/10.1145/335191.335372
Jiawei, H., Micheline, K., Jian, P.: Data Mining: Concepts and Techniques. Morgan Kaufmann (2011)
Kumar, S., Ghosh, J., Crawford, M.M.: Hierarchical fusion of multiple classifiers for hyperspectral data analysis. Pattern Anal. Appl. 5(2), 210–220 (2002)
Lei, H., Govindaraju, V.: Half-against-half multi-class support vector machines. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds.) MCS 2005. LNCS, vol. 3541, pp. 156–164. Springer, Heidelberg (2005)
Leonard, T., Hsu, J.S.: Bayesian Methods: An Analysis for Statisticians and Interdisciplinary Researchers. Cambridge University Press (2001)
Li, W., Han, J., Pei, J.: Cmar: Accurate and efficient classification based on multiple class-association rules. In: Proceedings of the 2001 IEEE International Conference on Data Mining, ICDM 2001, pp. 369–376. IEEE Computer Society, Washington, DC (2001), http://dl.acm.org/citation.cfm?id=645496.657866
Liu, B., Hsu, W., Ma, Y.: Integrating classification and association rule mining. In: Proc. KDD 1998 Conference (AAAI 1998), pp. 80–86 (1998)
Machov, K., Bark, F., Bednr, P.: A bagging method using decision trees in the role of base classifiers (2006)
Madzarov, G., Gjorgjevikj, D., Chorbev, I.: A multi-class svm classifier utilizing binary decision tree (2008)
Oza, N., Tumer, K.: Classifier ensembles: Select real-world applications. Information Fusion 9(1), 4–20 (2008), http://dx.doi.org/10.1016/j.inffus.2007.07.002
Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)
Rifkin, R.M., Klautau, A.: In defense of one-vs-all classification. Journal of Machine Learning Research 5, 101–141 (2004)
Schapire, R.E.: Using output codes to boost multiclass learning problems. In: Machine Learning: Proceedings of the Fourteenth International Conference (ICML1997) (1997)
Tax, D.M.J., Duin, R.P.W.: Using two-class classifiers for multiclass classification. In: ICPR (2), pp. 124–127 (2002)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Statistics for Engineering and Information Science. Springer (2000)
Vural, V., Dy, J.G.: A hierarchical method for multi-class support vector machines. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, p. 105. ACM, New York (2004), http://doi.acm.org/10.1145/1015330.1015427
Yin, X., Han, J.: Cpar: Classification based on predictive association rules (2003)
Zhang, G.P.: Neural networks for classification: A survey. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews 30(4), 451–462 (2000)
Zhou, Z.H.: Ensemble learning. In: Li, S.Z., Jain, A.K. (eds.) Encyclopedia of Biometrics, pp. 270–273. Springer US (2009), http://dblp.uni-trier.de/db/reference/bio/e.html#Zhou09
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Alshdaifat, E., Coenen, F., Dures, K. (2014). A Multi-path Strategy for Hierarchical Ensemble Classification. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2014. Lecture Notes in Computer Science(), vol 8556. Springer, Cham. https://doi.org/10.1007/978-3-319-08979-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-08979-9_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08978-2
Online ISBN: 978-3-319-08979-9
eBook Packages: Computer ScienceComputer Science (R0)