Abstract
This paper proposes the use of mutual information for feature selection in multi-label classification, a surprisingly almost not studied problem. A pruned problem transformation method is first applied, transforming the multi-label problem into a single-label one. A greedy feature selection procedure based on multidimensional mutual information is then conducted. Results on three databases clearly demonstrate the interest of the approach which allows one to sharply reduce the dimension of the problem and to enhance the performance of classifiers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Boutell, M., Luo, J., Shen, X., Brown, C.: Learning Multi-Label Scene Classification. Pattern Recogn. 37, 1757–1771 (2004)
Diplaris, S., Tsoumakas, G., Mitkas, P., Vlahavas, I.: Protein Classification with Multiple Algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–459. Springer, Heidelberg (2005)
Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multi-Label Classification of Music into Emotions. In: 9th International Conference on Music Information Retrieval (ISMIR 2008), Philadelphia, pp. 325–330 (2008)
Schapire, R.E., Singer, Y.: Boostexter: A Boosting-Based System for Text categorization. Machine Learning 39, 135–168 (2000)
Elisseeff, A., Weston, J.: A Kernel method for Multi-Labelled Classification. Advances in Neural Information Proceesing Systems 14, 681–687 (2001)
Zhang, M.-L., Zhou, Z.-H.: ML-KNN: A Lazy Learning Approach to Multi-Label Learning. Pattern Recogn. 40, 2038–2048 (2007)
Read, J.: A Pruned Problem Transformation Mathod for Multi-label Classification. In: New Zealand Computer Science Research Student Conference (NZCSRS 2008), pp. 143–150 (2008)
Guyon, I., Elisseeff, A.: An Introduction to Variable and Feature Selection. J. Mach. Lear. Res. 3, 1157–1182 (2003)
Shannon, C.E.: A mathematical Theory of Communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948)
Battiti, R.: Using Mutual Information for Selecting Features in Supervised Neural Net Learning. IEEE T. Neural. Networ. 5, 537–550 (1994)
Gomez-Verdejo, V., Verleysen, M., Fleury, J.: Information-Theoretic Feature Selection for Functional Data Classification. Neurocomputing 72, 3580–3589 (2009)
Kozachenko, L.F., Leonenko, N.: Sample Estimate of the Entropy of a Random Vector. Problems Inform. Transmission 23, 95–101 (1987)
Kraskov, A., Stögbauer, H., Grassberger, P.: Estimating Mutual Information. Phys. Rev. E 69, 066138 (2004)
Parzen, E.: On Estimation of a Probability Density Function and Mode. Ann. Math. Statist. 33, 1065–1076 (1962)
Benoudjit, N., François, D., Meurens, M., Verleysen, M.: Spectrophotometric Variable Selection by Mutual Information. Chemometr. Intell. Lab. 74, 243–251 (2004)
Francois, D., Rossi, F., Wertz, V., Verleysen, M.: Resampling Methods for Parameter-free and Robust Feature Selection with Mutual Information. Neurocomputing 70, 1276–1288 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Doquire, G., Verleysen, M. (2011). Feature Selection for Multi-label Classification Problems. In: Cabestany, J., Rojas, I., Joya, G. (eds) Advances in Computational Intelligence. IWANN 2011. Lecture Notes in Computer Science, vol 6691. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21501-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-21501-8_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21500-1
Online ISBN: 978-3-642-21501-8
eBook Packages: Computer ScienceComputer Science (R0)