A multi-objective algorithm for multi-label filter feature selection problem | Applied Intelligence Skip to main content
Springer Nature Link
Log in

A multi-objective algorithm for multi-label filter feature selection problem

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Feature selection is an important data preprocessing method before classification. Multi-objective optimization algorithms have been proved an effective way to solve feature selection problems. However, there are few studies on multi-objective optimization feature selection methods for multi-label data. In this paper, a multi-objective multi-label filter feature selection algorithm based on two particle swarms (MOMFS) is proposed. We use mutual information to measure the relevance between features and label sets, and the redundancy between features, which are taken as two objectives. In order to avoid Particle Swarm Optimization (PSO) from falling into the local optimum and obtaining a false Pareto front, we employ two swarms to optimize the two objectives separately and propose an improved hybrid topology based on particle’s fitness value. Furthermore, an archive maintenance strategy is introduced to maintain the distribution of archive. In order to study the effectiveness of the proposed algorithm, we select five multi-label evaluation criteria and perform experiments on seven multi-label data sets. MOMFS is compared with classic single-objective multi-label feature selection algorithms, multi-objective filter and wrapper feature selection algorithms. The experimental results show that MOMFS can effectively reduce the multi-label data dimension and perform better than other approaches on five evaluation criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Zhang M, Zhou Z (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    MATH  Google Scholar 

  2. Tahir MA, Kittler J, Bouridane A (2012) Multilabel classification using heterogeneous ensemble of multi-label classifiers. Pattern Recogn Lett 33(5):513–523

    Google Scholar 

  3. Stojanova D, Ceci M, Malerba D, Dzeroski S (2013) Using PPI network autocorrelation in hierarchical multi-label classification trees for gene function prediction. BMC Bioinformatics 14(1):285

    Google Scholar 

  4. Oramas S, Nieto O, Barbieri F et al (2017) Multi-label music genre classification from audio, text, and images using deep features. In: proceedings of the 18th International Society for Music Information Retrieval Conference, pp 23-30

  5. Wang J, Yang Y, Mao J et al (2016) Cnn-rnn: a unified framework for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294

  6. Zhao Z, Morstatter F, Sharma S et al (2010) Advancing feature selection research. ASU feature selection repository 1–28

  7. Rejer I, Twardochleb M (2018) Gamers’ involvement detection from EEG data with cGAAM-A method for feature selection for clustering. Expert Syst Appl 101:196–204

    Google Scholar 

  8. Lin S, Zhe Z, Xing L (2016) Particle swarm optimization-based feature selection in sentiment classification. Soft Comput 20(10):1–14

    Google Scholar 

  9. Mistry K, Zhang L, Neoh SC, Lim CP, Fielding B (2017) A micro-GA embedded PSO feature selection approach to intelligent facial emotion recognition. IEEE Transs Cybernetics 47(6):1496–1509

    Google Scholar 

  10. Fernandez-Lozano C, Seoane JA, Gestal M, Gaunt TR, Dorado J, Campbell C (2015) Texture classification using feature selection and kernel-based techniques. Soft Comput 19(9):2469–2480

    Google Scholar 

  11. Unler A, Murat A, Chinnam RB (2011) mr2 PSO: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inf Sci 181(20):4625–4641

    Google Scholar 

  12. Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626

    Google Scholar 

  13. Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybernetics 43(6):1656–1671

    Google Scholar 

  14. Tran, B, Xue B, Zhang M (2014) Overview of particle swarm optimisation for feature selection in classification. In: Asia-Pacific conference on simulated evolution and learning, pp 605–617

  15. Zhang Y, Wang S, Phillips P, Ji G (2014) Binary PSO with mutation operator for feature selection using decision tree applied to spam detection. Knowl-Based Syst 64(1):22–31

    Google Scholar 

  16. Dong H, Sun J, Li T, Li L et al (2018) An improved niching binary particle swarm optimization for feature selection. In: 2018 IEEE international conference on systems, man, and cybernetics, pp 3571-3577

  17. Yadav S, Ekbal A, Saha S (2017) Feature selection for entity extraction from multiple biomedical corpora: a PSO-based approach. Soft Comput 22(20):6881–6904

    Google Scholar 

  18. Teng X, Dong H, Sun J (2017) Co-evolutionary algorithm for feature selection. CAAI Trans Intelligent Syst 01:24–31

    Google Scholar 

  19. Feng T, Fu X, Zhang Y et al (2008) A genetic algorithm-based method for feature subset selection. Soft Comput 12(2):111–120

    Google Scholar 

  20. Dong H, Li T, Ding R, Sun J (2018) A novel hybrid genetic algorithm with granular information for feature selection and optimization. Appl Soft Comput 65:33–46

    Google Scholar 

  21. Khushaba RN, Al-Ani A, Al-Jumaily A (2011) Feature subset selection using differential evolution and a statistical repair mechanism. Expert Syst Appl 38(9):11515–11526

    Google Scholar 

  22. Xue B, Fu W, Zhang M (2014) Differential evolution (DE) for multi-objective feature selection in classification. In: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp 83–84

  23. Zorarpacı E, Özel SA (2016) A hybrid approach of differential evolution and artificial bee colony for feature selection. Expert Syst Appl 62:91–103

    Google Scholar 

  24. Li J, Cheng K, Wang S et al (2016) Feature selection: a data perspective. ACM Comput Surv 50(6):1–45

    Google Scholar 

  25. Xiong N (2002) A hybrid approach to input selection for complex processes. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 32(4):532–536

  26. Mao KZ (2004) Feature subset selection for support vector machines through discriminative function pruning analysis. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34(1):60–67

  27. Hsu CN, Huang H, Dietrich S (2002) The ANNIGMA-wrapper approach to fast feature selection for neural nets. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 32(2):207–212

  28. Zhang D, Chen S, Zhou Z (2008) Constraint score: a new filter method for feature selection with pairwise constraints. Pattern Recogn 41(5):1440–1451

    MATH  Google Scholar 

  29. Lei Y, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution. In: proceedings of the 20th international conference on machine learning, pp 856-863

  30. Dash M, Choi K, Scheuermann P et al (2002) Feature selection for clustering-a filter solution. In: IEEE International Conference on Data Mining, pp 115–122

  31. Hancer E, Xue B, Zhang M et al (2017) Pareto front feature selection based on artificial bee Colony optimization. Inf Sci 422:462–479

    Google Scholar 

  32. Spolaôr N, Lorena AC, Lee HD (2018) Feature selection via Pareto multi-objective genetic algorithms. Appl Artif Intell 31(32):1–28

    Google Scholar 

  33. Sikdar UK, Ekbal A, Saha S (2015) MODE: multiobjective differential evolution for feature selection and classifier ensemble. Soft Comput 19(12):3529–3549

    Google Scholar 

  34. Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybernetics 43(6):1656–1671

    Google Scholar 

  35. Jiménez F, Pérez-Sánchez H, Palma J, Sánchez G, Martínez C (2019) A methodology for evaluating multi-objective evolutionary feature selection for classification in the context of virtual screening. Soft Comput 23(18):8775–8800

    Google Scholar 

  36. Zhang Y, Gong D, Sun X, Guo YN (2017) A PSO-based multi-objective multi-label feature selection method in classification. Sci Rep 7(1):376

    Google Scholar 

  37. Khan M A, Ekbal A, Eneldo Loza Mencía et al (2017) Multi-objective optimisation-based feature selection for multi-label classification. In: International Conference on Applications of Natural Language to Information Systems, pp 38–41

  38. Zhang Y, Gong D, Rong M (2015) Multi-objective differential evolution algorithm for multi-label feature selection in classification. In: International Conference in Swarm Intelligence, pp 339–345

  39. Yin J, Tao T, Xu J (2015) A multi-label feature selection algorithm based on multi-objective optimization. In: International Joint Conference on Neural Networks, pp 1–7

  40. Zhan Z, Li J, Cao J et al (2013) Multiple populations for multiple objectives: a Coevolutionary technique for solving multiobjective optimization problems. IEEE Trans Syst Man Cybernetics Part B 43(2):445–463

    Google Scholar 

  41. Kennedy J (2002) Particle swarm optimization. In: proceedings of the ICNN95-international conference on neural networks, pp 1942-1948

  42. Eberhart RC, Shi Y (2007) Chapter four - evolutionary computation implementations. Computational intelligence 95-143

  43. Kalita DJ, Singh S (2020) SVM hyper-parameters optimization using quantized multi-PSO in dynamic environment. Soft Comput 24(2):1225–1241

    Google Scholar 

  44. Xuan T, Siarry P, Oulhadj H (2018) Integrating fuzzy entropy clustering with an improved PSO for MRI brain image segmentation. Appl Soft Comput 65:230–242

    Google Scholar 

  45. Zhao L, Wei J (2019) A nested particle swarm algorithm based on sphere mutation to solve bi-level optimization. Soft Comput 23(21):11331–11341

    Google Scholar 

  46. Guo Z, Quan L (2013) Financial time series forecasting using LPP and SVM optimized by PSO. Soft Comput 17(5):805–818

    Google Scholar 

  47. Kennedy J, Eberhart RC (1997) A discrete binary version of the particle swarm algorithm. In: International conference on systems, man, and cybernetics, pp 4104–4108

  48. Kennedy J (1999) Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In: proceedings of the 1999 congress on evolutionary computation, pp 1931-1938

  49. Liang J, Qin A, Suganthan PN et al (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295

    Google Scholar 

  50. Shannon CE (1948) A mathematical theory of communication. Bell Labs Tech J 27(3):379–423

    MathSciNet  MATH  Google Scholar 

  51. Battiti R (1994) Using mutual information for selecting features in supervised neural net learning. IEEE Trans Neural Netw 5(4):537–550

    Google Scholar 

  52. Liu H, Sun J, Lei L et al (2009) Feature selection with dynamic mutual information. Pattern Recogn 42(7):1330–1339

    MATH  Google Scholar 

  53. Bostani H, Sheikhan M (2015) Hybrid of binary gravitational search algorithm and mutual information for feature selection in intrusion detection systems. Soft Comput 21(9):1–18

    Google Scholar 

  54. Sluga D, Lotrič U (2017) Quadratic mutual information feature selection. Entropy 19(4):157–173

    MathSciNet  Google Scholar 

  55. Bidgoli, A. A., Rahnamayan, S., Ebrahimpour-Komleh, H. (2019). Opposition-based multi-objective binary differential evolution for multi-label feature selection. In: international conference on evolutionary multi-criterion optimization, pp 553–564

  56. Raquel CR, Naval Jr PC (2005) An effective use of crowding distance in multiobjective particle swarm optimization. In:Proceedings of the 7th annual conference on Genetic and evolutionary computation, pp 257–264

  57. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J et al (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12:2411–2414

    MathSciNet  MATH  Google Scholar 

  58. Zitzler E, Thiele L. Multiobjective Optimization Using Evolutionary Algorithms-A Comparative Study. In: International conference on parallel problem solving from nature, pp 292–301

  59. Spolaôr N, Cherman EA, Monard MC, Lee HD (2013) A comparison of multi-label feature selection methods using the problem transformation approach. Electronic Notes Theoretical Comput Sci 292:135–151

    Google Scholar 

  60. Peng H, Long F, Ding C (2005) Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Google Scholar 

  61. Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103

    Google Scholar 

  62. Huang R, Jiang W, Sun G (2018) Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recogn Lett 112:346–352

    Google Scholar 

  63. Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Pattern Recogn 95:136–150

    Google Scholar 

  64. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197

    Google Scholar 

  65. Coello CC, Lechuga MS (2002) MOPSO: a proposal for multiple objective particle swarm optimization. In: proceedings of the 2002 congress on evolutionary computation pp 1051-1056

Download references

Acknowledgments

This work was supported by the National Science Foundation of China (No. 61472095), Preparatory Research Foundation of Education Department of Heilongjiang (1354MSYYB003) and Research Foundation of Mudanjiang Normal University (YB2020010).

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Harbin Engineering University, Heilongjiang, 150001, China

    Hongbin Dong, Jing Sun, Tao Li & Xiaohang Sun

  2. Department of Computer Science and Technology, Mudanjiang Normal University, Heilongjiang, 157000, China

    Rui Ding

Authors
  1. Hongbin Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaohang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Sun.

Ethics declarations

Conflict of interest

Hongbin Dong declares that he has no conflict of interest. Jing Sun declares that she has no conflict of interest. Tao Li declares that he has no conflict of interest. Rui Ding declares that she has no conflict of interest. Xiaohang Sun declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, H., Sun, J., Li, T. et al. A multi-objective algorithm for multi-label filter feature selection problem. Appl Intell 50, 3748–3774 (2020). https://doi.org/10.1007/s10489-020-01785-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01785-2

Keywords

Search

Navigation