Abstract
In this article, we extend the idea of a priori knowledge in the form of detractor points presented recently for Support Vector Classification. We show that detractor points can belong to the new type of support vectors – training samples which lie outside a margin bounded region. We present the new application for a priori knowledge from detractor points – improving generalization performance of Support Vector Classification while reducing a complexity of a model by removing a bunch of support vectors. The experiments show that indeed the new type of a priori knowledge improves generalization performance of reduced models. The tests were performed on selected classification data sets, and on stock price data from public domain repositories.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Fung, G.M., Mangasarian, O.L., Shavlik, J.: Knowledge-based nonlinear kernel classifiers. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 102–113. Springer, Heidelberg (2003)
Fung, G.M., Mangasarian, O.L., Shavlik, J.W.: Knowledge-based support vector machine classifiers. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 521–528. MIT Press, Cambridge (2003)
Jean-Baptiste Pothin, C.R.: Incorporating prior information into support vector machines in the form of ellipsoidal knowledge sets (2006)
Joachims, T.: Transductive inference for text classification using support vector machines. In: ICML 1999: Proceedings of the Sixteenth International Conference on Machine Learning, pp. 200–209. Morgan Kaufmann Publishers Inc., San Francisco (1999)
Karasuyama, M., Takeuchi, I., Nakano, R.: Reducing svr support vectors by using backward deletion. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) KES 2008, Part III. LNCS (LNAI), vol. 5179, pp. 76–83. Springer, Heidelberg (2008)
Lauer, F., Bloch, G.: Incorporating prior knowledge in support vector machines for classification: A review. Neurocomput. 71(7-9), 1578–1594 (2008)
Le, Q.V., Smola, A.J., Gärtner, T.: Simpler knowledge-based support vector machines. In: ICML 2006: Proceedings of the 23rd International Conference on Machine Learning, pp. 521–528. ACM, New York (2006)
Libsvm data sets, http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
Lin, C.F., Wang, S.D.: Fuzzy support vector machines. IEEE Transaction on Neural Networks 13(2), 464–471 (2002)
Mangasarian, O.L., Wild, E.W.: Nonlinear knowledge-based classification. IEEE Transactions on Neural Networks 19(10), 1826–1832 (2008)
Orchel, M.: Support vector machines: Sequential multidimensional subsolver (sms). In: Dabrowski (professor), A. (ed.) Signal Processing: Algorithms, Architectures, Arrangements, and Applications SPA 2007. IEEE - The Institute of Electrical and Electronics Engineers Inc. Region 8 - Europe, Middle East and Africa. Chapter Circuits and Systems. Poland Section. Poznan University of Technology. Faculty of Computing Science and Management. Division of Signal Processing and Electronic Systems, pp. 135–140 (September 2007)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization, pp. 185–208. MIT Press, Cambridge (1999)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, Hoboken (1998)
Wang, L., Xue, P., Chan, K.L.: Incorporating prior knowledge into svm for image retrieval. In: ICPR 2004: 17th International Conference on Proceedings of the Pattern Recognition, vol. 2, pp. 981–984. IEEE Computer Society, Washington, DC (2004)
Wang, M., Yang, J., Liu, G.P., Xu, Z.J., Chou, K.C.: Weighted-support vector machines for predicting membrane protein types based on pseudo amino acid composition. Protein Engineering, Design & Selection 17(6), 509–516 (2004)
Wu, X., Srihari, R.: Incorporating prior knowledge with weighted margin support vector machines. In: KDD 2004: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 326–333. ACM, New York (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Orchel, M. (2011). Incorporating a Priori Knowledge from Detractor Points into Support Vector Classification. In: Dobnikar, A., Lotrič, U., Šter, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2011. Lecture Notes in Computer Science, vol 6594. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20267-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-20267-4_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20266-7
Online ISBN: 978-3-642-20267-4
eBook Packages: Computer ScienceComputer Science (R0)