Abstract
Hard margin support vector machines (HM-SVMs) have a risk of getting overfitting in the presence of the noise. Soft margin SVMs deal with this problem by the introduction of the capacity control term and obtain the state of the art performance. However, this disposal leads to a relatively high computational cost. In this paper, an alternative method, greedy stagewise algorithm, named GS-SVMs is presented to deal with the overfitting of HM-SVMs without the introduction of capacity control term. The most attractive property of GS-SVMs is that its computational complexity scales quadratically with the size of training samples in the worst case. Extensive empirical comparisons confirm the feasibility and validity GS-SVMs.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Haussler, D. (ed.) Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152. ACM Press, New York (1992)
Cotes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–279 (1995)
Tipping, M.: parse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211–244 (2001)
Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods — Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Computatio 13, 637–649 (2001)
Joachims, T.: Making large-scale SVM learning practical. In: Advances in Kernel Methods-Support Vector learning, pp. 169–184. MIT Press, Cambridge (1999)
Vishwanathan, S.V.N., Smola, A.J., Murty, M.N.: Simple SVM. In: Proceedings of the Twentieth International Conference on Machine Learning (2003)
Collobert, R., Bengio, S.: SVMTorch: Support Vector Machines for Large-Scale Regression Problems. Journal of Machine Learning Researc 1, 143–160 (2001)
Girosi, F.: An equivalence between sparse approximation and support vector machines. Neural Computation 10, 1455–1480 (1998)
Friedman, J.H.: Greedy Function Approximation: A gradient boosting machine. Annals of Statistics 29, 1189–1232 (2001)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bo, L., Wang, L., Jiao, L. (2005). Training Support Vector Machines Using Greedy Stagewise Algorithm. In: Ho, T.B., Cheung, D., Liu, H. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2005. Lecture Notes in Computer Science(), vol 3518. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11430919_73
Download citation
DOI: https://doi.org/10.1007/11430919_73
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26076-9
Online ISBN: 978-3-540-31935-1
eBook Packages: Computer ScienceComputer Science (R0)