Abstract
An empirical Bayes method to select basis functions and knots in multivariate adaptive regression spline (MARS) is proposed, which takes both advantages of frequentist model selection approaches and Bayesian approaches. A penalized likelihood is maximized to estimate regression coefficients for selected basis functions, and an approximated marginal likelihood is maximized to select knots and variables involved in basis functions. Moreover, the Akaike Bayes information criterion (ABIC) is used to determine the number of basis functions. It is shown that the proposed method gives estimation of regression structure that is relatively parsimonious and more stable for some example data sets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akaike H (1980) Likelihood and Bayes procedure. In: Bernardo JM et al (eds) Bayesian statistics. University Press, Valencia, pp 143–166
Brown BW, Hu MSJ (1980) Setting dose levels for the treatment of testicular cancer. In: Miller RG et al (eds) Biostatistics casebook. Wiley, New York, pp 113–152
Breiman L, Friedman JH, Olshen R, Stone CJ (1983) Classification and regression trees. Wadsworth, California
le Cessie S, van Houwelingen JC (1992) Ridge estimators in logistic regression. Appl Stat 41:191–201
Chambers JM, Hastie TJ (1992) Statistical models in S. Pacific Grove, California
Craven P, Wahba G (1979) Smoothing noisy data with spline functions. Numer Math 31:377–403
Davison AC (1986) Approximate predictive likelihood. Biometrika 73:323–332
Denison DGT, Holmes CC, Mallick BK, Smith AFM (2002) Bayesian methods for nonlinear classification and regression. Wiley, New York
Friedman JH (1991) Multivariate adaptive regression splines (with discussion). Ann Stat 19:1–141
Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232
Friedman JH, Silverman BW (1989) Flexible parsimonious smoothing and additive modeling. Technometrics 31:3–39
Good IJ (1965) The estimation of probabilities: an essay on modern Bayesian methods. MIT Press, Cambridge
Green PJ (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82:711–732
Green PJ, Silverman BW (1994) Nonparametric regression and generalized linear models. Chapman and Hall, London
Harrison D, Rubinfeld DL (1978) Hedonic prices and the demand for clean air. J Environ Econ Manage 5:81–102
Hoerl AE, Kennard RW (1971) Ridge regression: biased estimates for nonorthogonal problems. Technometrics 12:55–67
Hastie T, Tibshirani R (1990) Generalized additive models. Chapman and Hall, London
Hastie T, Tibshirani R (2000) Bayesian backfitting. Stat Sci 15:196–223
Lin X, Zhang D (1999) Inference in generalized additive mixed models by using smoothing splines. J R Stat Soc B 61:381–400
McCullagh P, Nelder JA (1989) Generalized linear models, 2nd edn. Chapman and Hall, London
O’Sullivan F, Yandell BS, Raynor WJ (1986) Automatic smoothing of regression functions in generalized linear models. J Am Stat Assoc 81:96–103
Salford Systems (2001) MARS User’s Guide.
Simonoff JS (1996) Smoothing methods in statistics. Springer, New York
Tierney L, Kadane JB (1986) Accurate approximations for posterior moments and marginal densities. J Am Stat Assoc 81:82–86
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sakamoto, W. MARS: selecting basis functions and knots with an empirical Bayes method. Computational Statistics 22, 583–597 (2007). https://doi.org/10.1007/s00180-007-0075-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-007-0075-7