Search for the best decision rules with the help of a probabilistic estimate | Annals of Mathematics and Artificial Intelligence Skip to main content
Springer Nature Link
Log in

Search for the best decision rules with the help of a probabilistic estimate

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

The problem of how to find the best decision rule in the course of a search, based on sample set analysis, is considered. Specifically the problem of selecting the best subset of regressors is highlighted. In the considered formulation the problem is an important case of how to learn a dependence by examples.

The concept of Predictive Probabilistic Estimate (PPE) is introduced and its properties are discussed. Asymptotic properties of PPE are based on Vapnik-Chervonenkis theory of uniform convergence of a set of sample estimates. Finite-sample-size properties of PPE demonstrate how PPE takes into account the presence of a search process and the complexity of a regression formula, while estimating quality of fit. Some practical and model examples are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. M.A. Aitkin, Simultaneous inference and the choice of variable subsets, Technometrics 16 (1974) 221–227.

    Google Scholar 

  2. V.L. Brailovsky and I.Yu. Brailovsky, Method of predicting solar flares with a function of many variables, English translation: Sov. Phys., Doklady (1972) 335–337.

  3. K.N. Berk, Comparing subset regression procedures, Technometrics 20 (1978) 1–6.

    Google Scholar 

  4. L. Breiman, J.H. Friedman, R.A. Olsen and C.J. Stone,Classification and Regression Trees (Wadsworth, Belmont, CA, 1984).

    Google Scholar 

  5. A. Blumer, A. Ehrenfencht, D. Haussler and M. Warmuth, Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension,Proc. 18th Annual ACM Symp. on Theory of Computing, Berkeley, CA (May 1986) pp. 273–282.

  6. V.L. Brailovsky, A predictive probabilistic estimate for selecting subsets of regressor variables, Ann. N.Y. Acad. Sci. 491 (1987) 233–244.

    Google Scholar 

  7. V.L. Brailovsky, On the use of a predictive probabilistic estimate for selecting best decision rules in the course of search,Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Ann Arbor, MI (1988) pp. 469–477.

  8. V.L. Brailovsky, A probabilistic estimate of clustering,Proc. 10th Int. Conf. on Pattern Recognition, Atlantic City, NJ (June 1990) vol. 1, pp. 953–956.

  9. G. Diehr and D.R. Hoflin, Approximating the distribution of the sampleR 2 in best subset regressions, Technometrics 16 (1974) 317–330.

    Google Scholar 

  10. N. Draper and H. Smith,Applied Regression Analysis (Wiley, 1966).

  11. V.F. Flack and P.C. Chang, Frequency of selecting noisy variables, The Amer. Statist. 41 (1987) 84–86.

    Google Scholar 

  12. A.B. Forsythe, L. Engleman, P.R.A. May and R. Jennrich, A stopping rule for variable selection in multiple regression analysis, J. Amer. Statist. Assoc. 68 (1973) 75–77.

    Google Scholar 

  13. J.W. Gorman and R.J. Torman, Selection of variables for fitting equations to data, Technometrics 8 (1966) 27–51.

    Google Scholar 

  14. A. Hald,Statistical Theory with Engineering Applications (Wiley, 1952).

  15. A.L. Luntz and V.L. Brailovsky, Evaluation of attributes obtained in statistical decision rules, English translation: Eng. Cybern. 3 (1967) 98–109.

    Google Scholar 

  16. A.J. Miller, Selection of subsets of regression variables, J. Roy. Stat. Soc. A147 (1984) part 2.

  17. R.G. Miller, Statistical prediction by discriminant analysis, Meteorol. Monog., Am. Meteorol. Soc. 4 (1962) 21.

    Google Scholar 

  18. I.Sh. Pinsker, The chaotization principle and its application in data analysis, in:Models, Algorithms, Decision Making, ed. I. Pinsker (Nauka, Moscow, 1979) (in Russian).

    Google Scholar 

  19. A.C. Rencher and F.C. Pun, Inflation ofR 2 in best subset regression, Technometrics 22 (1980) 49–53.

    Google Scholar 

  20. J. Rissanen, Modelling by shortest data description, Automatica 14 (1978) 465–471.

    Google Scholar 

  21. J. Rissanen, A universal prior for integers and estimation by minimum description length, Ann. Statist. 11 (2) (1983) 416–431.

    Google Scholar 

  22. G.A.F. Seber,Linear Regresion Analysis (Wiley, 1977).

  23. V.N. Vapnik and A.Ja. Chervonenkis,Theory of Pattern Recognition (Nauka, Moscow, 1974) (in Russian).

    Google Scholar 

  24. V.N. Vapnik,Estimation of Dependencies, Based on Empirical Data (Springer, 1982).

Download references

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Raymond and Beverly Sackler Faculty of Exact Sciences, Tel-Aviv University, 69978, Ramat-Aviv, Israel

    Victor L. Brailovsky

Authors
  1. Victor L. Brailovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brailovsky, V.L. Search for the best decision rules with the help of a probabilistic estimate. Ann Math Artif Intell 4, 249–267 (1991). https://doi.org/10.1007/BF01531059

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01531059

Keywords

Search

Navigation