Incremental one-class classifier based on convex–concave hull | Pattern Analysis and Applications Skip to main content
Springer Nature Link
Log in

Incremental one-class classifier based on convex–concave hull

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

One subject that has been considered less is a binary classification on data streams with concept drifting in which only information of one class (target class) is available for learning. Well-known methods such as SVDD and convex hull have tried to find the enclosed boundary around target class, but their high complexity makes them unsuitable for large data sets and also online tasks. This paper presents a novel online one-class classifier adapted to the streaming data. Considering time complexity, an incremental convex–concave hull classification method, called ICCHC, is proposed which can significantly reduce the computational time and expand the target class boundary. Also, it can be adapted to the gradual concept drift. Evaluations have been conducted on seventeen real-world data sets by hold-out validation. Also, noise analysis has been carried out. The results of the experiments have been compared with the state-of-the-art methods, which show the superiority of ICCHC regarding the accuracy, precision, and recall metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Cui L, Shi Y (2014) A method based on one-class SVM for news recommendation. Procedia Comput Sci 31:281–290

    Google Scholar 

  2. Moradi M, Hamidzadeh J (2019) Ensemble-based Top-k recommender system considering incomplete data. J AI Data Min 7(3):393–402

    Google Scholar 

  3. Bowen RM (2016) Online Novelty Detection System: One-Class Classification of Systemic Operation

  4. Manevitz LM, Yousef M (2001) One-class SVMs for document classification. J Mach Learn Res 2(Dec):139–154

    MATH  Google Scholar 

  5. Chen Y, Hu B, Keogh E, Batista GE (2013) DTW-D: time series semi-supervised learning from a single example. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 383–391

  6. Pauwels EJ, Ambekar O (2011) One class classification for anomaly detection: support vector data description revisited. In: Industrial conference on data mining. Springer, pp 25–39

  7. Tax DMJ (2001) One-class classification; concept-learning in the absence of counter-examples. Delft University of Technology, Delft

    Google Scholar 

  8. Duda RO, Hart PE, Stork DG (2012) Pattern classification. Wiley, Hoboken

    MATH  Google Scholar 

  9. Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, Oxford

    MATH  Google Scholar 

  10. Van Hulle MM (2012) Self-organizing maps. In: Rozenberg G, Bäck T, Kok JN (eds) Handbook of natural computing. Springer, Berlin, Heidelberg, pp 585–622. https://doi.org/10.1007/978-3-540-92910-9_19’

    Chapter  Google Scholar 

  11. Zeng M, Yang Y, Luo S, Cheng J (2016) One-class classification based on the convex hull for bearing fault detection. Mech Syst Signal Process 81:274–293

    Google Scholar 

  12. Wang W, Zhang B, Wang D, Jiang Y, Qin S, Xue L (2016) Anomaly detection based on probability density function with Kullback–Leibler divergence. Signal Process 126:12–17

    Google Scholar 

  13. Hamidzadeh J, Monsefi R, Yazdi HS (2015) IRAHC: instance reduction algorithm using hyperrectangle clustering. Pattern Recogn 48(5):1878–1889

    MATH  Google Scholar 

  14. Moghaddam VH, Hamidzadeh J (2016) New Hermite orthogonal polynomial kernel and combined kernels in support vector machine classifier. Pattern Recogn 60:921–935

    MATH  Google Scholar 

  15. Hamidzadeh J, Moradi M (2018) Improved one-class classification using filled function. Appl Intell 48:1–17

    Google Scholar 

  16. Utkin LV, Zhuk YA (2017) An one-class classification support vector machine model by interval-valued training data. Knowl Based Syst 120:43–56

    Google Scholar 

  17. Zeng M, Yang Y, Cheng J (2016) A generalized Mitchell–Dem’yanov–Malozemov algorithm for one-class support vector machine. Knowl Based Syst 109:17–24

    Google Scholar 

  18. Bhattacharya BK (1982) Application of computational geometry to pattern recognition problems. Ph.d. thesis, School of Computer Science, McGill University

  19. Toussaint G (1978) The convex hull as a tool in pattern recognition. In: AFOSR workshop in communication theory and applications

  20. Amini S, Homayouni S, Safari A, Darvishsefat AA (2018) Object-based classification of hyperspectral data using Random Forest algorithm. Geo-Spat inf Sci 21(2):127–138

    Google Scholar 

  21. Chang Z, Cao J, Zhang Y (2018) A novel image segmentation approach for wood plate surface defect classification through convex optimization. J For Res 29(6):1789–1795

    Google Scholar 

  22. Chazelle B (1993) An optimal convex hull algorithm in any fixed dimension. Discrete Comput Geom 10(1):377–409

    MathSciNet  MATH  Google Scholar 

  23. Ruano A, Khosravani HR, Ferreira PM (2015) A randomized approximation convex hull algorithm for high dimensions. IFAC-PapersOnLine 48(10):123–128

    Google Scholar 

  24. Chau AL, Li X, Yu W (2013) Convex and concave hulls for classification with support vector machine. Neurocomputing 122:198–209

    Google Scholar 

  25. Kodell RL, Zhang C, Siegel ER, Nagarajan R (2012) Selective voting in convex-hull ensembles improves classification accuracy. Artif Intell Med 54(3):171–179

    Google Scholar 

  26. Zeng M, Yang Y, Zheng J, Cheng J (2015) Maximum margin classification based on flexible convex hulls. Neurocomputing 149:957–965

    Google Scholar 

  27. Khan L, Fan W (2012) Tutorial: data stream mining and its applications. In: International conference on database systems for advanced applications. Springer, pp 328–329

  28. Shalev-Shwartz S (2007) Online learning: theory, algorithms, and applications. Thesis submitted for the degree of “Doctor of Philosophy” (Submitted to the Senate of the Hebrew University July 2007, This work was carried out under the supervision of Prof. Yoram Singer)

  29. Jiang Y, Shang J, Liu Y (2010) Maximizing customer satisfaction through an online recommendation system: a novel associative classification model. Decis Support Syst 48(3):470–479

    Google Scholar 

  30. Sagar B, Singh P, Mallika S (2016) Online transaction fraud detection techniques: a review of data mining approaches. In: 2016 3rd International conference on computing for sustainable global development (INDIACom). IEEE, pp 3756–3761

  31. Olszewski D (2012) A probabilistic approach to fraud detection in telecommunications. Knowl Based Syst 26:246–258

    Google Scholar 

  32. Olszewski D (2014) Fraud detection using self-organizing map visualizing the user profiles. Knowl Based Syst 70:324–334

    Google Scholar 

  33. Cohen Y, Gordon D, Hendler D (2017) Early detection of spamming accounts in large-Scale service provider networks. Knowl Based Syst 142:241–255

    Google Scholar 

  34. Wang B, Jones GJ, Pan W (2006) Using online linear classifiers to filter spam emails. Pattern Anal Appl 9(4):339–351

    MathSciNet  Google Scholar 

  35. Dilmen E, Beyhan S (2017) A novel online LS-SVM approach for regression and classification. IFAC-PapersOnLine 50(1):8642–8647

    Google Scholar 

  36. Ding S, Mirza B, Lin Z, Cao J, Lai X, Nguyen TV, Sepulveda J (2018) Kernel based online learning for imbalance multiclass classification. Neurocomputing 277:139–148

    Google Scholar 

  37. Suárez-Cetrulo AL, Cervantes A (2017) An online classification algorithm for large scale data streams: iGNGSVM. Neurocomputing 262:67–76

    Google Scholar 

  38. Gensler A, Sick B (2018) Performing event detection in time series with SwiftEvent: an algorithm with supervised learning of detection criteria. Pattern Anal Appl 21(2):543–562

    MathSciNet  Google Scholar 

  39. Soleimani-B H, Lucas C, Araabi BN (2012) Fast evolving neuro-fuzzy model and its application in online classification and time series prediction. Pattern Anal Appl 15(3):279–288

    MathSciNet  Google Scholar 

  40. Hamidzadeh J, Moradi M (2020) Enhancing data analysis: uncertainty-resistance method for handling incomplete data. Appl Intell 50(1):74–86

    Google Scholar 

  41. Aggarwal CC (2014) A survey of stream classification algorithms. Data classification: algorithms and applications. Springer, New York, USA, pp 245–268

    Google Scholar 

  42. Sousa R, Gama J (2018) Multi-label classification from high-speed data streams with adaptive model rules and random rules. Prog Artif Intell 7:1–11

    Google Scholar 

  43. Gu X, Angelov PP (2018) Semi-supervised deep rule-based approach for image classification. Appl Soft Comput 68:53–68

    Google Scholar 

  44. Verdecia-Cabrera A, Blanco IF, Carvalho AC (2018) An online adaptive classifier ensemble for mining non-stationary data streams. Intell Data Anal 22(4):787–806

    Google Scholar 

  45. Ramírez-Gallego S, García S, Herrera F (2018) Online entropy-based discretization for data streaming classification. Future Gener Comput Syst 86:59–70

    Google Scholar 

  46. Lobo JL, Del Ser J, Bilbao MN, Perfecto C, Salcedo-Sanz S (2018) DRED: an evolutionary diversity generation method for concept drift adaptation in online learning environments. Appl Soft Comput 68:693–709

    Google Scholar 

  47. Feng L-R, Liu C-M, Lai C-C (2018) Probabilistic reverse nearest neighbors on uncertain data streams. In: 2018 7th International symposium on next generation electronics (ISNE). IEEE, pp 1–4

  48. Shao X, Zhang M, Meng J (2018) Data stream clustering and outlier detection algorithm based on shared nearest neighbor density. In: 2018 International conference on intelligent transportation, big data and smart city (ICITBS). IEEE, pp 279–282

  49. Chatzigeorgakidis G, Karagiorgou S, Athanasiou S, Skiadopoulos S (2018) FML-kNN: scalable machine learning on Big Data using k-nearest neighbor joins. J Big Data 5(1):4

    Google Scholar 

  50. Duda P, Jaworski M, Rutkowski L (2018) Convergent time-varying regression models for data streams: tracking concept drift by the recursive parzen-based generalized regression neural networks. Int J Neural Syst 28(02):1750048

    Google Scholar 

  51. Prasetyo T, Amar S, Arendra A, Zami MZ (2018) On-line tool wear detection on DCMT070204 carbide tool tip based on noise cutting audio signal using artificial neural network. In: Journal of physics: conference series, vol 1. IOP Publishing, p 012144

  52. van Rijn JN, Holmes G, Pfahringer B, Vanschoren J (2018) The online performance estimation framework: heterogeneous ensemble learning for data streams. Mach Learn 107(1):149–176

    MathSciNet  MATH  Google Scholar 

  53. Brzezinski D, Stefanowski J, Nienkötter A, Jiang X, Last M, Stoliar M, Friedman M, Cornelisse R, Choenni S, Munir M (2018) Ensemble classifiers for imbalanced and evolving data streams. Ser Mach Percept Artif Intell 83(1):44–68

    Google Scholar 

  54. Krawczyk B, Cano A (2018) Online ensemble learning with abstaining classifiers for drifting and noisy data streams. Appl Soft Comput 68:677–692

    Google Scholar 

  55. Bifet A, Holmes G, Pfahringer B, Kirkby R, Gavaldà R (2009) New ensemble methods for evolving data streams. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 139–148

  56. Domingos P, Hulten G (2000) Mining high-speed data streams. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 71–80

  57. Hulten G, Spencer L, Domingos P (2001) Mining time-changing data streams. In: Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 97–106

  58. Hashemi S, Yang Y (2009) Flexible decision tree for data stream classification in the presence of concept change, noise and missing values. Data Min Knowl Disc 19(1):95–131

    MathSciNet  Google Scholar 

  59. Hashemi S, Yang Y, Mirzamomen Z, Kangavari M (2008) Adapted one-versus-all decision trees for data stream classification. IEEE Trans Knowl Data Eng 5:624–637

    Google Scholar 

  60. Bifet A, Read J, Holmes G, Pfahringer B (2018) Streaming data mining with massive online analytics (MOA). Ser Mach Percept Artif Intell 83(1):1–25

    Google Scholar 

  61. Kang JH, Park CH, Kim SB (2016) Recursive partitioning clustering tree algorithm. Pattern Anal Appl 19(2):355–367

    MathSciNet  Google Scholar 

  62. Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ, Williamson RC (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471

    MATH  Google Scholar 

  63. Sabzekar M, Yazdi HS, Naghibzadeh M (2012) Relaxed constraints support vector machine. Expert Syst 29(5):506–525

    Google Scholar 

  64. Zhang Y, Chi Z-X (2008) A Fuzzy support vector classifier based on Bayesian optimization. Fuzzy Optim Decis Mak 7(1):75–86

    MATH  Google Scholar 

  65. Preparata FP, Shamos MI (1985) Introduction. In: Gries D, Schneider F (eds) Computational geometry. Springer, New York, pp 1–35. https://doi.org/10.1007/978-1-4612-1098-6_1

    Chapter  MATH  Google Scholar 

  66. Liu Z, Liu J, Pan C, Wang G (2009) A novel geometric approach to binary classification based on scaled convex hulls. IEEE Trans Neural Netw 20(7):1215–1220

    Google Scholar 

  67. Casale P, Pujol O, Radeva P (2014) Approximate polytope ensemble for one-class classification. Pattern Recogn 47(2):854–864

    MATH  Google Scholar 

  68. Bennett KP, Bredensteiner EJ (2000) Duality and geometry in SVM classifiers. In: ICML. pp 57–64

  69. Takahashi T, Kudo M (2010) Margin preserved approximate convex hulls for classification. In: 2010 20th International conference on pattern recognition (ICPR). IEEE, pp 4052–4055

  70. Pal S, Hattacharya S (2007) Neurocomputing model for computation of an approximate convex hull of a set of points and spheres. IEEE Trans Neural Netw 18(2):600–605

    Google Scholar 

  71. Ding S, Nie X, Qiao H, Zhang B (2017) A fast algorithm of convex hull vertices selection for online classification. IEEE Trans Neural Netw Learn Syst 29:792–806

    MathSciNet  Google Scholar 

  72. Casale P, Pujol O, Radeva P (2011) Approximate convex hulls family for one-class classification. In: International workshop on multiple classifier systems. Springer, pp 106–115

  73. Castillo E, Peteiro-Barral D, Berdiñas BG, Fontenla-Romero O (2015) Distributed one-class support vector machine. Int J Neural Syst 25(07):1550029

    Google Scholar 

  74. Kemmler M, Rodner E, Wacker E-S, Denzler J (2013) One-class classification with Gaussian processes. Pattern Recogn 46(12):3507–3518

    Google Scholar 

  75. Katz A, Thrift P (1993) Hybrid neural network classifiers for automatic target detection. Expert Syst 10(4):243–250

    Google Scholar 

  76. Zhang Y, Li X, Orlowska M (2008) One-class classification of text streams with concept drift. In: IEEE international conference on data mining workshops, 2008. ICDMW’08. IEEE, pp 116–125

  77. Zhu X, Wu X, Zhang C (2009) Vague one-class learning for data streams. In: Ninth IEEE international conference on data mining, 2009. ICDM’09. IEEE, pp 657–666

  78. Zhang D, Cai L, Wang Y, Zhang L (2010) A learning algorithm for one-class data stream classification based on ensemble classifier. In: 2010 International conference on computer application and system modeling (ICCASM). IEEE, pp V2-596–V592-600

  79. Li X, Liu B (2003) Learning to classify texts using positive and unlabeled data. In: IJCAI, vol 2003. pp 587–592

  80. Liu B, Lee WS, Yu PS, Li X (2002) Partially supervised classification of text documents. In: ICML. pp 387–394

  81. Zhu X, Ding W, Philip SY, Zhang C (2011) One-class learning and concept summarization for data streams. Knowl Inf Syst 28(3):523–553

    Google Scholar 

  82. Liu B, Xiao Y, Philip SY, Cao L, Zhang Y, Hao Z (2014) Uncertain one-class learning and concept summarization learning on uncertain data streams. IEEE Trans Knowl Data Eng 26(2):468–484

    Google Scholar 

  83. Gao K (2015) Online one-class SVMs with active-set optimization for data streams. In: 2015 IEEE 14th International conference on machine learning and applications (ICMLA). IEEE, pp 116–121

  84. Saunier N, Midenet S (2013) Creating ensemble classifiers through order and incremental data selection in a stream. Pattern Anal Appl 16(3):333–347

    MathSciNet  Google Scholar 

  85. Dokur Z (2009) Respiratory sound classification by using an incremental supervised neural network. Pattern Anal Appl 12(4):309

    MathSciNet  Google Scholar 

  86. Afzal A, Asharaf S (2017) Deep kernel learning in core vector machines. Pattern Anal Appl 21:1–9

    Google Scholar 

  87. Wu T, Liang Y, Varela R, Wu C, Zhao G, Han X (2016) Self-adaptive SVDD integrated with AP clustering for one-class classification. Pattern Recogn Lett 84:232–238

    Google Scholar 

  88. Jiang Y, Wang Y, Luo H (2015) Fault diagnosis of analog circuit based on a second map SVDD. Analog Integr Circ Sig Process 85(3):395–404

    Google Scholar 

  89. Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. In: Burges CJC, Smola AJ (eds) Advances in neural information processing systems, pp 409–415

  90. Krawczyk B, Woźniak M (2015) One-class classifiers with incremental learning and forgetting for data streams with concept drift. Soft Comput 19(12):3387–3400

    Google Scholar 

  91. Bicego M, Figueiredo MA (2009) Soft clustering using weighted one-class support vector machines. Pattern Recogn 42(1):27–32

    MATH  Google Scholar 

  92. Das B, Cook DJ, Krishnan NC, Schmitter-Edgecombe M (2016) One-class classification-based real-time activity error detection in smart homes. IEEE J Sel Top Signal Process 10(5):914–923

    Google Scholar 

  93. Krawczyk B, Woźniak M (2015) Incremental weighted one-class classifier for mining stationary data streams. J Comput Sci 9:19–25

    Google Scholar 

  94. Zhou X, Zhang X, Zhang B (2015) An incremental convex hull algorithm based online support vector regression. In: Control conference (CCC), 2015 34th Chinese. IEEE, pp 8220–8225

  95. Wang D, Qiao H, Zhang B, Wang M (2013) Online support vector machine based on convex hull vertices selection. IEEE Trans Neural Netw Learn Syst 24(4):593–609

    Google Scholar 

  96. Tax DM, Duin RP (2004) Support vector data description. Mach Learn 54(1):45–66

    MATH  Google Scholar 

  97. Liu B, Xiao Y, Cao L, Hao Z, Deng F (2013) SVDD-based outlier detection on uncertain data. Knowl Inf Syst 34:1–22

    Google Scholar 

  98. Yin G, Zhang Y-T, Li Z-N, Ren G-Q, Fan H-B (2014) Online fault diagnosis method based on incremental support vector data description and extreme learning machine with incremental output structure. Neurocomputing 128:224–231

    Google Scholar 

  99. Sadeghi R, Hamidzadeh J (2016) Automatic support vector data description. Soft Comput 22:1–12

    Google Scholar 

  100. Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17(2–3):191–209

    MATH  Google Scholar 

  101. Zadeh LA (1996) Fuzzy sets. In: Klir GJ, Yuan B (eds) Fuzzy sets, fuzzy logic, and fuzzy systems: selected papers by Lotfi A Zadeh. World Scientific, pp 394–432

  102. Pawlak Z (1982) Rough sets. Int J Parallel Prog 11(5):341–356

    MATH  Google Scholar 

  103. Radzikowska AM, Kerre EE (2002) A comparative study of fuzzy rough sets. Fuzzy Sets Syst 126(2):137–155

    MathSciNet  MATH  Google Scholar 

  104. Verbiest N, Cornelis C, Herrera F (2013) FRPS: a fuzzy rough prototype selection method. Pattern Recogn 46(10):2770–2782

    MATH  Google Scholar 

  105. Sinha D, Laplante P (2004) A rough set-based approach to handling spatial uncertainty in binary images. Eng Appl Artif Intell 17(1):97–110

    Google Scholar 

  106. Wang QH, Li JR (2004) A rough set-based fault ranking prototype system for fault diagnosis. Eng Appl Artif Intell 17(8):909–917

    MathSciNet  Google Scholar 

  107. Bárány I (1982) A generalization of Carathéodory’s theorem. Discrete Math 40(2–3):141–152

    MathSciNet  MATH  Google Scholar 

  108. Vapnik V (2013) The nature of statistical learning theory. Springer, Berlin

    MATH  Google Scholar 

  109. Bartlett PL, Mendelson S (2002) Rademacher and Gaussian complexities: Risk bounds and structural results. J Mach Learn Res 3(Nov):463–482

    MathSciNet  MATH  Google Scholar 

  110. Zhu X, Wu X (2004) Class noise vs attribute noise: a quantitative study. Artif Intell Rev 22(3):177–210

    MATH  Google Scholar 

  111. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27

    Google Scholar 

  112. Sheskin DJ (2003) Handbook of parametric and nonparametric statistical procedures. CRC Press, Boca Raton

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering and Information Technology, Sadjad University of Technology, Mashhad, Iran

    Javad Hamidzadeh & Mona Moradi

Authors
  1. Javad Hamidzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mona Moradi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javad Hamidzadeh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamidzadeh, J., Moradi, M. Incremental one-class classifier based on convex–concave hull. Pattern Anal Applic 23, 1523–1549 (2020). https://doi.org/10.1007/s10044-020-00876-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-020-00876-7

Keywords

Search

Navigation