A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods

@article{Burman1989ACS,
  title={A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods},
  author={Prabir Burman},
  journal={Biometrika},
  year={1989},
  volume={76},
  pages={503-514},
  url={https://api.semanticscholar.org/CorpusID:16701030}
}
  • P. Burman
  • Published 1 September 1989
  • Mathematics, Computer Science
  • Biometrika
Concepts of v-fold cross-validation and repeated learning-testing methods are introduced here and are computationally much less expensive than ordinary cross- validation and can be used in its place in many problems.
View via Publisher
ece.uvic.ca
675 Citations
Highly Influential Citations
28
Background Citations
93
Methods Citations
244
Results Citations
15

Tables from this paper

675 Citations

Some Issues in Cross-Validation

A new type of cross- validation is proposed here for model selection problems when the data is generated by a stationary process, which is an off-shoot of both ordinary cross-validation and v-fold cross- validation.

Cross-validation as the objective function for variable-selection techniques

An assessment of ten-fold and Monte Carlo cross validations for time series forecasting

Experimental results, using time series of the NN3 tournament, found that Monte Carlo cross validation is more stable than ten-fold cross validation for selecting the best forecasting model.

Multiple predicting K-fold cross-validation for model selection

This study proposes a new CV method that uses folds of the data for model validation, while the other fold is for model construction, and provides predicted values for each observation to reduce variation in the assessment due to the averaging.

Estimation of prediction error by using K-fold cross-validation

This paper investigates two families that connect the training error and K-fold cross-validation, which has a downward bias and has an upward bias.

A survey of cross-validation procedures for model selection

This survey intends to relate the model selection performances of cross-validation procedures to the most recent advances of model selection theory, with a particular emphasis on distinguishing empirical statements from rigorous theoretical results.

Consistent Cross Validation with Stable Learners

A debiased version of the K-fold is proposed which is consistent for any uniformly stable learner and applies to the problem of model selection and demonstrates empirically the usefulness of the promoted approach on real world datasets.

Cross-validation

This text is a survey on cross-validation. We define all classical cross-validation procedures, and we study their properties for two different goals: estimating the risk of a given estimator, and…

On optimal data split for generalization estimation and model selection

The theoretical basics of various cross-validation techniques are described with the purpose of reliably estimating the generalization error and optimizing the model structure for reliably estimating a single location parameter.

An empirical comparison of $$V$$V-fold penalisation and cross-validation for model selection in distribution-free regression

Cases in which VFCV and $$V$$V-fold penalisation provide poor estimates of the risk, respectively, are highlighted, and a modified penalisation technique is introduced to reduce the estimation error.
...

13 References

An alternative method of cross-validation for the smoothing of density estimates

An alternative method of cross-validation, based on integrated squared error, recently also proposed by Rudemo (1982), is derived, and Hall (1983) has established the consistency and asymptotic optimality of the new method.

Cross‐Validatory Choice and Assessment of Statistical Predictions

SUMMARY A generalized form of the cross-validation criterion is applied to the choice and assessment of prediction using the data-analytic concept of a prescription. The examples used to illustrate…

The Predictive Sample Reuse Method with Applications

A recently devised method of prediction based on sample reuse techniques that is most useful in low structure data paradigms that involve minimal assumptions is presented.

Classification and regression trees

This chapter discusses tree classification in the context of medicine, where right Sized Trees and Honest Estimates are considered and Bayes Rules and Partitions are used as guides to optimal pruning.

Generalized $L-, M-$, and $R$-Statistics

Abstract : A class of statisticss generalizing U-statistics and L-statistics, and containing other varieties of statistics as well, such as trimmed U-statistics, is studied. Using the differentiable…

Estimating Optimal Transformations for Multiple Regression and Correlation.

Abstract In regression analysis the response variable Y and the predictor variables X 1 …, Xp are often replaced by functions θ(Y) and O1(X 1), …, O p (Xp ). We discuss a procedure for estimating…

Optimal Bandwidth Selection in Nonparametric Regression Function Estimation

On considere des estimateurs du noyau d'une fonction de regression multivariable et une regle de selection selon la largeur de bande formulee en terme de validation croisee

Jackknife Approximations to Bootstrap Estimates

Let T be an estimate of the form Tn = T(F ) where F is the nn n' n sample cdf of n iid observations and T is a locally quadratic functional defined on cdf's. Then, the normalized jackknife estimates…

Approximation Theorems of Mathematical Statistics

Preliminary Tools and Foundations. The Basic Sample Statistics. Transformations of Given Statistics. Asymptotic Theory in Parametric Inference. U--Statistics. Von Mises Differentiable Statistical…