Abstract
We propose a learning framework to address multiclass challenges, namely visualization, scalability and performance. We focus on supervised problems by presenting an approach that uses prior information about training labels, manifold learning and support vector machines (SVMs).
We employ manifold learning as a feature reduction step, nonlinearly embedding data in a low dimensional space using Isomap (Isometric Mapping), enhancing geometric characteristics and preserving the geodesic distance within the manifold. Structured SVMs are used in a multiclass setting with benefits for final multiclass classification in this reduced space. Results on a text classification toy example and on ISOLET, an isolated letter speech recognition problem, demonstrate the remarkable visualization capabilities of the method for multiclass problems in the severely reduced space, whilst improving SVMs baseline performance.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Heidelberg (1999)
Dumais, S., Platt, J., Heckerman, D.: Inductive Learning Algorithms and Representations for Text categorisation. In: ACM Conf. Information Knowledge Management, pp. 148–155 (1998)
Crammer, K., Singer, Y.: On the Algorithmic Implementation of Multi-class kernel-based vector machines. Journal Machine Learning Research 2, 265–292 (2002)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: Int. Conf. Machine Learning, pp. 104–111 (2004)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Large Margin Methods for Structured and Interdependent Output Variables. Journal Machine Learning Research 6, 1453–1484 (2005)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 5500(290), 2319–2323 (2000)
Kim, H., Park, H., Zha, H.: Distance Preserving Dimension Reduction for Manifold Learning. In: Int. Conf. Data Mining, vol. II, pp. 1147–1151 (2007)
Navarro, D., Lee, M.D.: Spatial Visualization of Document Similarity, Defence Human Factors. Special Interest Group Meeting (2001)
Zhang, D., Chen, X., Lee, W.: Text Classification with Kernels on the Multinomial Manifold. In: ACM SIGIR - Special Interest Group on Information Retrieval, pp. 266–273 (2005)
Silva, C., Ribeiro, B.: Text Classification on Embedded Manifolds. In: Geffner, H., Prada, R., Machado Alexandre, I., David, N. (eds.) IBERAMIA 2008. LNCS (LNAI), vol. 5290, pp. 272–281. Springer, Heidelberg (2008)
Collins, M.: Parameter estimation for statistical parsing models: Theory and practice of distribution-free methods. In: IWPT - International Workshop on Parsing Technologies (2001)
Jolliffe, I.T.: Principal Component Analysis. Springer, Heidelberg (1986)
Comon, P.: Independent Component Analysis: a New Concept? Signal Processing 36(3), 287–314 (1994)
Cox, T., Cox, M.: Multidimensional Scaling. Chapman & Hall, London (1994)
Duraiswami, R., Raykar, V.C.: The Manifolds of Spatial Hearing. In: ICASSP 2005, vol. III, pp. 285–288 (2005)
Geng, X., Zhan, D., Zhou, Z.: Supervised Nonlinear Dimensionality Reduction for Visualization and Classification. IEEE Transactions Systems, Man, and Cybernetics – Part B 35(6), 1098–1107 (2005)
Specht, D.: A General Regression Neural Network. IEEE Transactions on Neural Networks 2(6), 568–576 (1991)
Fanty, M., Cole, R.: Spoken letter recognition. In: Advances in Neural Information Processing Systems, vol. 3 (1991)
van Rijsbergen, C.: Information Retrieval. Butterworths Ed. (1979)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Silva, C., Ribeiro, B. (2009). Improving Visualization, Scalability and Performance of Multiclass Problems with SVM Manifold Learning. In: Kolehmainen, M., Toivanen, P., Beliczynski, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2009. Lecture Notes in Computer Science, vol 5495. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04921-7_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-04921-7_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04920-0
Online ISBN: 978-3-642-04921-7
eBook Packages: Computer ScienceComputer Science (R0)