Evaluation of the Cleft-Overstep Algorithm for Linear Regression Analysis | SpringerLink
Skip to main content
Springer Nature Link
Log in

Evaluation of the Cleft-Overstep Algorithm for Linear Regression Analysis

  • Conference paper
  • First Online:
Computational Collective Intelligence (ICCCI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12496))

Included in the following conference series:

Abstract

Optimization algorithms have been applied to improve the learning speed of machine learning. In the backpropagation phase, the weights of the neural network will be updated in each epoch with the purpose that the difference between the actual output and predicted one will be reached to a value smaller than a predefined number epsilon. In essence, the backpropagation process only uses the idea of optimization algorithm to formulate a weighted update formula, rather than a complete optimization problem. The enhancement of machine learning speed is meaningful when the application of artificial intelligence is rising. The cleft-overstep optimal algorithm has been introduced since the 1990s and implemented in optimal control, which has not been popularized and evaluated in other fields. This paper aims to use the idea of cleft-overstep method, also apply and program probing in linear regression analysis - a simple machine learning algorithm. Results were compared with classic gradient descent method based on the known problem and showed a significant learning speed improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ruszczyński, A.: Nonlinear Optimization, vol. 13. Princeton University Press, Princeton (2006)

    Google Scholar 

  2. Zouggar, S.T., Adla, A.: Optimization techniques for machine learning. In: Kulkarni, A.J., Satapathy, S.C. (eds.) Optimization in Machine Learning and Applications. AIS, pp. 31–50. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-0994-0_3

    Chapter  Google Scholar 

  3. Ezquerro, J.A., Grau-Sánchez, M., Grau, A., et al.: On iterative methods with accelerated convergence for solving systems of nonlinear equations. J. Optim. Theory Appl. 151, 163–174 (2011)

    Article  MathSciNet  Google Scholar 

  4. Ruder, S.: An overview of gradient descent optimization algorithms, pp. 1–14 (2016). https://arxiv.org/abs/1609.04747

  5. Botev, A., Lever, G., Barber, D.: Nesterov’s accelerated gradient and momentum as approximations to regularised update descent. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1899–1903 (2017)

    Google Scholar 

  6. Zhang, N., Lei, D., Zhao, J.F.: An improved Adagrad gradient descent optimization algorithm. In: Proceedings Chinese Automation Congress, CAC 2018, no. 3, pp. 2359–2362. IEEE (2019)

    Google Scholar 

  7. Park, H., Lee, K.: Adaptive natural gradient method for learning of stochastic neural networks in mini-batch mode. Appl. Sci. 9, 4568 (2019)

    Google Scholar 

  8. Manh, N.V.: Diss. Doc. Sc.: Optimizing methods of uncertainty control systems. MPEI, Moscow (1999)

    Google Scholar 

  9. Manh, N.V., Tri, B.M.: Bisector method solves the unconstraines optimization problem. Math J. XV(4) (1987). (in Vietnamese)

    Google Scholar 

  10. Manh, N.V., Tri, B.M.: Method of “cleft-overstep” by perpendicular direction for solving unconstraines nonlinear optimization problem. Acta Mathematica Vietnamica 15(2), 73–83 (1990)

    MathSciNet  MATH  Google Scholar 

  11. Manh, N.V., Tri, B.M.: An extreme algorithm based on a combination of the “cleft-overstep” principle and the principle of spatial stretching. J. Sci. Technol. (12), 1–6 (1996). (in Vietnamese)

    Google Scholar 

  12. Pal, M., Bharati, P.: Introduction to correlation and linear regression analysis. Applications of Regression Techniques, pp. 1–18. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-9314-3_1

    Chapter  MATH  Google Scholar 

  13. Perron, P., Yamamoto, Y.: Pitfalls of two-step testing for changes in the error variance and coefficients of a linear regression model. Econometrics 7, 22 (2019)

    Google Scholar 

  14. Jon, D.: Appendix D matrix calculus. In: Convex Optimization & Euclidean Distance Geometry. Meboo Publishing (2018). https://ccrma.stanford.edu/~dattorro/matrixcalc.pdf

  15. Scikit-learn (2019). https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_blobs.html

  16. Hung, P.D., Su, N.T., Diep, V.T.: Surface classification of damaged concrete using deep convolutional neural network. Pattern Recogn. Image Anal. 29(4), 676–687 (2019). https://doi.org/10.1134/S1054661819040047

    Article  Google Scholar 

  17. Nam, N.T, Hung, P.D.: Padding methods in convolutional sequence model: an application in Japanese handwriting recognition. In: Proceedings of the 3rd International Conference on Machine Learning and Soft Computing (ICMLSC 2019), pp. 138–142. Association for Computing Machinery, New York (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. FPT University, Hanoi, Vietnam

    Duy Hung Phan & Le Dinh Huynh

Authors
  1. Duy Hung Phan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Le Dinh Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Duy Hung Phan .

Editor information

Editors and Affiliations

  1. Department of Applied Informatics, Wrocław University of Science and Technology, Wroclaw, Poland

    Ngoc Thanh Nguyen

  2. Thua Thien Hue Center of Information Technology, Hue, Vietnam

    Bao Hung Hoang

  3. Vietnam - Korea University of Information and Communication Technology, University of Da Nang, Da Nang, Vietnam

    Cong Phap Huynh

  4. Department of Computer Engineering, Yeungnam University, Gyeungsan, Korea (Republic of)

    Dosam Hwang

  5. Department of Applied Informatics, Wrocław University of Science and Technology, Wroclaw, Poland

    Bogdan Trawiński

  6. Department of Information Systems, University of Münster, Münster, Germany

    Gottfried Vossen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Phan, D.H., Huynh, L.D. (2020). Evaluation of the Cleft-Overstep Algorithm for Linear Regression Analysis. In: Nguyen, N.T., Hoang, B.H., Huynh, C.P., Hwang, D., Trawiński, B., Vossen, G. (eds) Computational Collective Intelligence. ICCCI 2020. Lecture Notes in Computer Science(), vol 12496. Springer, Cham. https://doi.org/10.1007/978-3-030-63007-2_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63007-2_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63006-5

  • Online ISBN: 978-3-030-63007-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation