Abstract
We study optimization of the regularized least-squares classification algorithm, and proposes an early-stopping training procedure. Different from previous empirical training methods which separate model selection and parameter learning into two stages, the proposed method performs the two processes simultaneously and thus reduces the training time significantly. We carried out a series of evaluations on text categorization tasks. The experimental results verified the effectiveness of our training method, with comparable classification accuracy and significantly improved running speed over conventional training methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bishop, C.: Pattern Recognition and Machine Learning. Springer (2008)
Vapnik, V.: Statistical Learning Theory. John Wiley and Sons (1998)
Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press (2002)
Tikhonov, A., Arsenin, V.: Solutions of Ill-Posed Problems. Winston and Sons (1977)
Poggio, T., Girosi, F.: Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks. Science 247, 978–982 (1990)
Platt, J.: Fast Training of Support Vector Machines Using Sequential Minimal Optimization. In: Advances in Kernel Methods – Support Vector Learning. MIT Press (1999)
Kimeldorf, G., Wahba, G.: Some Results on Thebycheffian Spline Functions. J. Math. Anal. Appl. 33, 82–95 (1971)
Girosi, F., Jones, M., Poggio, T.: Regularization Theory and Neural Networks Architectures. Neural Comput. 7, 219–269 (1995)
Schölkopf, B., Herbrich, R., Smola, A.J.: A Generalized Representer Theorem. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 416–426. Springer, Heidelberg (2001)
Fung, G., Mangasarian, O.: Proximal Support Vector Machine Classifiers. In: Proceedings of KDD 2001 (2001)
Rifkin, R.: Everything Old is New Again: A Fresh Look at Historical Approaches in Machine Learning. PhD thesis. MIT (2002)
Suykens, J., Gestel, T., Brabanter, J., B.D.Moor, Vandewalle, J.: Least Squares Support Vector Machines. World Scientific Publishing (2002)
Golub, G., Loan, C.V.: Matrix Computations. John Hopkins Press (1996)
Engl, H., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer Academic Publishers (1996)
Bauer, F., Pereverzev, S., Rosasco, L.: On Regularization Algorithms in Learning Theory. Journal of Complexity 23, 52–72 (2007)
Li, W., Lee, K.H., Leung, K.S.: Large-scale RLSC Learning Without Agony. In: Proceedings of the 24th Annual International Conference on Machine Learning, pp. 529–536. ACM (2007)
Sun, X., Pitsianis, N.: A Matrix Version of the Fast Multipole Method. SIAM Review 43, 289–300 (2001)
Joachims, T.: Training Linear SVMs in Linear Time. In: Proceedings of SIGKDD 2006. ACM (2006)
Li, W., Lee, K.H., Leung, K.S.: Generalized Regularized Least-Squares Learning with Predefined Features in a Hilbert Space. In: Advances in Neural Information Processing Systems, vol. 19, pp. 881–888. MIT Press (2006)
Li, W., Leung, K.S., Lee, K.H.: Generalizing the Bias Term of Support Vector Machines. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 919–924. AAAI (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Li, W. (2014). Early-Stopping Regularized Least-Squares Classification. In: Zeng, Z., Li, Y., King, I. (eds) Advances in Neural Networks – ISNN 2014. ISNN 2014. Lecture Notes in Computer Science(), vol 8866. Springer, Cham. https://doi.org/10.1007/978-3-319-12436-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-12436-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12435-3
Online ISBN: 978-3-319-12436-0
eBook Packages: Computer ScienceComputer Science (R0)