Abstract
The problem of automatically tuning multiple parameters for pattern recognition Support Vector Machines (SVMs) is considered. This is done by minimizing some estimates of the generalization error of SVMs using a gradient descent algorithm over the set of parameters. Usual methods for choosing parameters, based on exhaustive search become intractable as soon as the number of parameters exceeds two. Some experimental results assess the feasibility of our approach for a large number of parameters (more than 100) and demonstrate an improvement of generalization performance.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Bengio. Y. (2000). Gradient-based optimization of hyper-parameters. Neural Computation, 12:8.
Bonnans J. F. & Shapiro, A. (2000). Perturbation analysis of optimization problems. Berlin: Springer-Verlag.
Chapelle, O. & Vapnik, V. (1999). Model selection for support vector machines. In Advances in neural information processing systems.
Cortes, C. & Vapnik, V. (1995). Support vector networks. Machine Learning, 20, 273-297.
Cristianini, N., Campbell, C., & Shawe-Taylor, J. (1999). Dynamically adapting kernels in support vector machines. In Advances in neural information processing systems.
Cristianini, N. & Shawe-Taylor, J. (2000). An introduction to support vector machines. Cambridge, MA: Cambridge University Press.
Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M. L., Downing, J. R., Caligiuri, M. A., Bloomfield, C. D., & Lander, E. S. (1999). Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science, 286, 531-537.
Heisele, B., Poggio, T., & Pontil, M. (2000). Face detection in still gray images. AI Memo 1687, Massachusetts Institute of Technology.
Jaakkola, T. S. & Haussler, D. (1999). Probabilistic kernel regression models. In Proceedings of the 1999 Conference on AI and Statistics.
Joachims, T. (2000). Estimating the generalization performance of a svm efficiently. In Proceedings of the International Conference on Machine Learning. San Mateo, CA: Morgan Kaufman.
Larsen, J., Svarer, C., Andersen, L. N., & Hansen, L. K. (1998). Adaptive regularization in neural network modeling. In G. B. Orr & K. R. Müller (Eds.). Neural networks: Trick of the trade. Berlin: Springer.
Luntz, A. & Brailovsky, V. (1969). On estimation of characters obtained in statistical procedure of recognition. Technicheskaya Kibernetica, 3, (in Russian).
Lütkepohl, H. (1996). Handbook of matrices. New York: Wiley & Sons.
Opper, M. & Winther, O. (2000). Gaussian processes and svm: Mean field and leave-one-out. In A. J. Smola, P. L. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.). Advances in large margin classifiers (pp. 311-326). Cambridge, MA: MIT Press.
Platt, J. (2000). Probabilities for support vector machines. In A. Smola, P. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.). Advances in large margin classifiers. Cambridge, MA: MIT Press.
Rätsch, G., Onoda, T., & Müller, K.-R. (2001). Soft margins for AdaBoost. Machine Learning, 42:3, 287-320.
Serre, T., Heisele, B., Mukherjee, S., & Poggio, T. (2000). Feature selection for face detection. AI Memo 1697, Massachusetts Institute of Technology.
Vapnik, V. (1995). The nature of statistical learning theory. Berlin: Springer.
Vapnik, V. (1998). Statistical learning theory. New York: John Wiley & Sons.
Vapnik, V. & Chapelle, O. (2000). Bounds on error expectation for support vector machines: Neural Computation, 12:9.
Wahba, G., Lin, Y., & Zhang, H. (2000). Generalized approximate crossvalidation for support vector machines: Another way to look at marginlike quantities. In A. Smola, P. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.). Advances in large margin classifiers (pp. 297-309). Cambridge, MA: MIT Press.
Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., & Vapnik, V. (2000). Feature selection for support vector machines. In Advances in neural information processing systems.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Chapelle, O., Vapnik, V., Bousquet, O. et al. Choosing Multiple Parameters for Support Vector Machines. Machine Learning 46, 131–159 (2002). https://doi.org/10.1023/A:1012450327387
Issue Date:
DOI: https://doi.org/10.1023/A:1012450327387