Supervised and semi-supervised twin parametric-margin regularized extreme learning machine | Pattern Analysis and Applications Skip to main content
Springer Nature Link
Log in

Supervised and semi-supervised twin parametric-margin regularized extreme learning machine

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Twin extreme learning machine (TELM) has attracted considerable attention and achieved great success in the machine learning field. However, its performance will be severely affected when outliers exist in the dataset since TELM does not consider heteroscedasticity in practical applications. To improve the performance of TELM, a novel learning framework called twin parametric-margin extreme learning machine (TPMELM) was proposed. Further, to enhance the classification performance of our TPMELM in a semi-supervised learning setting, a Laplacian TPMELM (Lap-TPMELM) was developed by introducing manifold regularization into TPMELM. Using the geometric information of the marginal distribution embedded in unlabeled samples, Lap-TPMELM can effectively construct a more reasonable classifier. The TPMELM and Lap-TPMELM are suitable for many situations, especially when the data has heteroscedastic error structure. Moreover, the TPMELM and Lap-TPMELM are helpful in clarifying theoretical interpretation of parameters which control the bounds on proportions of support vectors and boundary errors. An efficient technique (successive over-relaxation, SOR) is applied in TPMELM and Lap-TPMELM, respectively. Experimental results show the effectiveness and reliability of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://www.ntu.edu.sg/home/egbhuang/.

  2. https://www.mathworks.com/.

  3. http://archive.ics.uci.edu/ml/datasets.html.

  4. http://people.cs.uchicago.edu/vikass/manifoldregularization.html.

  5. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php.

  6. https://cs.nyu.edu/~roweis/data.html.

  7. http://yann.lecun.com/exdb/mnist/.

References

  1. Huang G, Zhu Q, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501

    Article  Google Scholar 

  2. Huang G, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163

    Article  Google Scholar 

  3. Huang G, Huang G, Song S, You K (2015) Trends in extreme learning machines: a review. Neural Netw Off J Int Neural Netw Soc 61:32–48

    Article  Google Scholar 

  4. Ma J, Wen Y, Yang L (2018) Lagrangian supervised and semi-supervised extreme learning machine. Appl Intell 49:303–318

    Article  Google Scholar 

  5. Shen J, Ma J (2019) Sparse twin extreme learning machine with \(\varepsilon\)-insensitive zone pinball loss. IEEE Access 7:112067–112078

    Article  Google Scholar 

  6. Ma J, Yuan C (2019) Adaptive safe semi-supervised extreme machine learning. IEEE Access 7:76176–76184

    Article  Google Scholar 

  7. Ma J, Yang L, Wen Y, Sun Q (2019) Twin minimax probability extreme learning machine for pattern recognition. Knowl Based Syst. https://doi.org/10.1016/j.knosys.2019.06.014

    Article  Google Scholar 

  8. Zhang W, Ji H, Liao G, Zhang Y (2015) A novel extreme learning machine using privileged information. Neurocomputing 168:823–828

    Article  Google Scholar 

  9. Zhang Y, Wu J, Cai Z, Zhang P, Chen L (2016) Memetic extreme learning machine. Pattern Recogn 58:135–148

    Article  Google Scholar 

  10. Ding XJ, Lan Y, Zhang ZF et al (2017) Optimization extreme learning machine with \(\nu\) regularization. Neurocomputing 261:11–19

    Article  Google Scholar 

  11. Huang G, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B (Cybern) 42:513–529

    Article  Google Scholar 

  12. Cherkassky V (1997) The nature of statistical learning theory. IEEE Trans Neural Netw 8(6):1564

    Article  Google Scholar 

  13. Jayadeva Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910

    Article  Google Scholar 

  14. Peng X (2010) \(\nu\)-twin support vector machine ( \(\nu\)-TSVM) classifier and its geometric algorithms. Inf Sci 180(20):3863–3875

    Article  MathSciNet  Google Scholar 

  15. Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22:962–968

    Article  Google Scholar 

  16. Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw Off J Int Neural Netw Soc 35:46–53

    Article  Google Scholar 

  17. Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recogn 46:305–316

    Article  Google Scholar 

  18. Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81

    Article  Google Scholar 

  19. Peng X, Wang Y, Xu D (2013) Structural twin parametric-margin support vector machine for binary classification. Knowl Based Syst 49:63–72

    Article  Google Scholar 

  20. Wan Y, Song S, Huang G, Li S (2017) Twin extreme learning machines for pattern classification. Neurocomputing 260:235–244

    Article  Google Scholar 

  21. Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12:1207–1245

    Article  Google Scholar 

  22. Hao P (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw Off J Int Neural Netw Soc 23(1):60–73

    Article  Google Scholar 

  23. Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44:2678–2692

    Article  Google Scholar 

  24. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MathSciNet  MATH  Google Scholar 

  25. Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233

    MATH  Google Scholar 

  26. Melacci S, Belkin M (2009) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184

    MathSciNet  MATH  Google Scholar 

  27. Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):130

    MATH  Google Scholar 

  28. Yang Z, Xu Y (2016) Laplacian twin parametric-margin support vector machine for semi-supervised classification. Neurocomputing 171:325–334

    Article  Google Scholar 

  29. Huang G, Song S, Gupta JN, Wu C (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 44:2405–2417

    Article  Google Scholar 

  30. Liu B, Xia S, Meng F, Zhou Y (2014) Manifold regularized extreme learning machine. Neural Comput Appl 27:255–269

    Article  Google Scholar 

  31. Mangasarian OL, Musicant DR (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037

    Article  Google Scholar 

  32. Qi Z, Tian Y, Shi Y (2015) Successive overrelaxation for laplacian support vector machine. IEEE Trans Neural Netw Learn Syst 26(4):674–683

    Article  MathSciNet  Google Scholar 

  33. Chen WJ, Shao YH, Ye YF (2013) Improving Lap-TSVM with successive overrelaxation and differential evolution. Procedia Comput Sci 17:33–40

    Article  Google Scholar 

  34. Gao X, Lu T, Liu P, Lu Q (2014) A soil moisture classification model based on SVM used in agricultural WSN. In: IEEE 7th joint international information technology and artificial intelligence conference, pp 432–436

  35. Fernández Pierna JA, Lecler B, Conzen JP, Niemoeller A, Baeten V, Dardenne P (2011) Comparison of various chemometric approaches for large near infrared spectroscopic data of feed and feed products. Anal Chim Acta 705(1–2):0-34

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (Grant Nos. 11471010 and 11271367).

Author information

Authors and Affiliations

  1. College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China

    Jun Ma

Authors
  1. Jun Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jun Ma.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, J. Supervised and semi-supervised twin parametric-margin regularized extreme learning machine. Pattern Anal Applic 23, 1603–1626 (2020). https://doi.org/10.1007/s10044-020-00880-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-020-00880-x

Keywords