Exploring instance correlations with local discriminant model for multi-label feature selection | Applied Intelligence Skip to main content
Springer Nature Link
Log in

Exploring instance correlations with local discriminant model for multi-label feature selection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-Label Learning (MLL) is a classification task in which each instance may be associated with two or more class labels simultaneously. With the arrival of information explosion era, MLL is confronted with the curse of high-dimensional data. Feature selection is an effective technique that has been used to reduce dimensionality. Previous Multi-Label Feature Selection (MLFS) approaches only lay emphasis on the fitting relations between the feature space and the corresponding label space while neglecting the instance correlations. To remedy the deficiency, we present a novel MLFS method by Exploring Instance Correlations with Local Discriminant model (EICLD). More specifically, we first construct a local set for each instance, and use a local discriminant model for each local set to explore instance correlations. Then, we further integrate the local models of all instances to guarantee the global instance correlations. Finally, l2,1-norm is introduced into loss function and the regularization of feature weight matrix respectively to facilitate feature selection process. An optimization algorithm is designed for handling the proposed objective function. The paper is compared with seven representative approaches on twelve data sets. The results validate the superiority of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Awadallah MA, Al-Betar MA, Hammouri AI, Alomari OA (2020) Binary JAYA algorithm with adaptive mutation for feature selection. Arab J Sci Eng 45(12):10875–10890

  2. Alweshah M, Al Khalaileh S, Gupta BB, Almomani A, Hammouri AI, Al-Betar MA (2020) The monarch butterfly optimization algorithm for solving feature selection problems. Neural Comput Appl:1–15

  3. Al-Salemi B, Ayob M, Noah S A M (2018) Feature ranking for enhancing boosting-based multi-label text categorization. Expert Syst Appl 113:531–543

    Article  Google Scholar 

  4. Battiti R (1994) Using mutual information for selecting features in supervised neural net learning. IEEE Trans Neural Netw 5(4):537–550

    Article  Google Scholar 

  5. Bidgoli A A, Ebrahimpour-Komleh H, Rahnamayan S (2019) A novel multi-objective binary differential evolution algorithm for multi-label feature selection. Proceedings of the 2019 IEEE Congress on Evolutionary Computation, pp 1588–1595

  6. Bidgoli A A, Ebrahimpour-Komleh H, Rahnamayan S (2020) Reference-point-based multi-objective optimization algorithm with opposition-based voting scheme for multi-label feature selection. Inf Sci 547:1–17

    Article  MathSciNet  MATH  Google Scholar 

  7. Braytee A, Wei L, Catchpoole D R, Kennedy P J (2017) Multi-label feature selection using correlation information. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp 1649–1656

  8. Cai X, Nie F, Huang H (2013) Exact top-k feature selection via l2,0-norm constraint. Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, pp 1240–1246

  9. Cai Z, Zhu W (2018) Multi-label feature selection via feature manifold learning and sparsity regularization. Int J Mach Learn Cybern 9(8):1321–1334

    Article  Google Scholar 

  10. Chen L, Zhang H, Lu J, Thung K, Aibaidula A, Liu L, Chen S, Jin L, Wu J, Wang Q, Zhou L, Shen D (2018) Multi-label nonlinear matrix completion with transductive multi-task feature selection for joint MGMT and IDH1 status prediction of patient with high-grade gliomas. IEEE Trans Med Imaging 37(8):1775– 1787

    Article  Google Scholar 

  11. Chen S, Zhang Y, Ding C, Zhang J, Luo B (2019) Extended adaptive Lasso for multi-class and multi-label feature selection. Knowl-Based Syst 173:28–36

    Article  Google Scholar 

  12. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(1):1–30

    MathSciNet  MATH  Google Scholar 

  13. Dong H, Sun J, Sun X, Ding R (2020) A many-objective feature selection for multi-label classification. Knowl-Based Syst 208:106456

    Article  Google Scholar 

  14. Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S (2021) Multi-label feature selection with constraint regression and adaptive spectral graph. Knowl-Based Syst 212:106621

    Article  Google Scholar 

  15. Fürnkranz J, Hüllermeier E, Mencía E L, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153

    Article  MATH  Google Scholar 

  16. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(3):1157–1182

    MATH  Google Scholar 

  17. Hou C, Nie F, Yi D, Wu Y (2011) Feature selection via joint embedding learning and sparse regression. Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, pp 1324–1329

  18. Hu J, Li Y, Gao W, Zhang P (2020) Robust multi-label feature selection with dual-graph regularization. Knowl-Based Syst 203:106126

    Article  Google Scholar 

  19. Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Pattern Recogn 104:107344

    Article  Google Scholar 

  20. Huang H, Liu H (2020) Feature selection for hierarchical classification via joint semantic and structural information of labels. Knowl-Based Syst 195:105655

    Article  Google Scholar 

  21. Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323

    Article  Google Scholar 

  22. Huang J, Li G, Huang Q, Wu X (2017) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889

    Article  Google Scholar 

  23. Huang R, Jiang W, Sun G (2018) Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recogn Lett 112:346–352

    Article  Google Scholar 

  24. Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp 1627–1633

  25. Karagoz G N, Yazici A, Dokeroglu T, Cosar A (2020) A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data. Int J Mach Learn Cybern 1:1–19

    Google Scholar 

  26. Kashef S, Nezamabadi-pour H, Nikpour B (2018) Multilabel feature selection: a comprehensive review and guiding experiments. Wiley Interdiscip Rev Data Min Knowl Discov 8(2):e1240

    Article  Google Scholar 

  27. Lee J, Kim D (2013) Feature selection for multi-label classification using multivariate mutual information. Pattern Recogn Lett 34:349–57

    Article  Google Scholar 

  28. Lee J, Kim D (2015) Fast multi-label feature selection based on information-theoretic feature ranking. Pattern Recogn 48:2761– 71

    Article  MATH  Google Scholar 

  29. Lee J, Kim D (2015) Mutual information-based multi-label feature selection using interaction information. Expert Syst Appl 42:2013–2025

    Article  Google Scholar 

  30. Lee J, Yu I, Park J, Kim D (2019) Memetic feature selection for multilabel text categorization using label frequency difference. Inf Sci 485:263–280

    Article  Google Scholar 

  31. Li Y, Lin Y, Liu J, Weng W, Shi Z, Wu S (2018) Feature selection for multi-label learning based on kernelized fuzzy rough sets. Neurocomputing 318:271–286

    Article  Google Scholar 

  32. Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103

    Article  Google Scholar 

  33. Lin Y, Hu Q, Liu J, Li J, Wu X (2017) Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Trans Fuzzy Syst 25(6):1491–1507

    Article  Google Scholar 

  34. Lin Y, Li Y, Wang C, Chen J (2018) Attribute reduction for multi-label learning with fuzzy rough set. Knowl-Based Syst 152:51–61

    Article  Google Scholar 

  35. Liu J, Li Y, Weng W, Zhang J, Chen B, Wu S (2020) Feature selection for multi-label learning with streaming label. Neurocomputing 387:268–278

    Article  Google Scholar 

  36. Liu J, Lin Y, Li Y, Weng W, Wu S (2018) Online multi-label streaming feature selection based on neighborhood rough set. Pattern Recogn 84:273–287

    Article  Google Scholar 

  37. Liu J, Lin Y, Wu S, Wang C (2018) Online multi-label group feature selection. Knowl-Based Syst 143:42–57

    Article  Google Scholar 

  38. Ma Z, Nie F, Yang Y, Uijlings J R, Sebe N (2012) Web image annotation via subspace-sparsity collaborated feature selection. IEEE Trans Multimed 14(4):1021–1030

    Article  Google Scholar 

  39. Mishra N K, Singh P K (2020) FS-MLC: Feature Selection for multi-label classification using clustering in feature space. Inf Process Manag 57(4):102240

  40. Nie F, Huang H, Cai X, Ding C H (2010) Efficient and robust feature selection via joint l2,1-norms minimization. Proceedings of the Twenty-Third International Conference on Neural Information Processing Systems, pp 1813–1821

  41. Noorie Z, Afsari F (2020) Sparse feature selection: relevance, redundancy and locality structure preserving guided by pairwise constraints. Appl Soft Comput 87:105956

    Article  Google Scholar 

  42. Pang T, Nie F, Han J, Li X (2018) Efficient feature selection via l2,0-norm constrained sparse regression. IEEE Trans Knowl Data Eng 31(5):880–893

    Article  Google Scholar 

  43. Paniri M, Dowlatshahi M B, Nezamabadi-pour H (2020) MLACO: A multi-label feature selection algorithm based on ant colony optimization. Knowl-Based Syst 192:105285

    Article  Google Scholar 

  44. Pereira R B, Plastino A, Zadrozny B, Merschmann L H (2018) Categorizing feature selection methods for multi-label classification. Artif Intell Rev 49(1):57–78

    Article  Google Scholar 

  45. Qi G-J, Hua X-S, Rui Y, Tang J, Mei T, Zhang H-J (2007) Correlative multi-label video annotation. Proceedings of the Fifteenth ACM international conference on Multimedia, pp 17–26

  46. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333

    Article  MathSciNet  Google Scholar 

  47. Seo W, Kim D, Lee J (2019) Generalized information-theoretic criterion for multi-label feature selection. IEEE Access 7:122854–122863

    Article  Google Scholar 

  48. Sha Z -C, Liu Z -M, Ma C, Chen J (2020) Feature selection for multi-label classification by maximizing full-dimensional conditional mutual information. Appl Intell:1–15

  49. Shang R, Xu K, Shang F, Jiao L (2020) Sparse and low-redundant subspace learning-based dual-graph regularized robust feature selection. Knowl-Based Syst 187:104830

    Article  Google Scholar 

  50. Sun Z, Zhang J, Dai L, Li C, Zhou C, Xin J, Li S (2019) Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing 329:447–456

    Article  Google Scholar 

  51. Tawhid M A, Ibrahim A M (2020) Feature selection based on rough set approach, wrapper approach, and binary whale optimization algorithm. Int J Mach Learn Cybern 11(3):573–602

    Article  Google Scholar 

  52. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multi- label classification of music into emotions. Proc ISMIR 8:325–330

  53. Wang C, Lin Y, Liu J (2019) Feature selection for multi-label learning with missing labels. Appl Intell 49:3027–3042

    Article  Google Scholar 

  54. Wang W, Dai Q, Li F, Xiong Y, Wei D (2020) MLCDForest: multi-label classification with deep forest in disease prediction for long non-coding RNAs. Briefings in Bioinformatics

  55. Wen J, Xu Y, Li Z, Ma Z, Xu Y (2018) Inter-class sparsity based discriminative least square regression. Neural Netw 102:36–47

    Article  MATH  Google Scholar 

  56. Weng W, Chen C -L, Wu S -X, Li Y -W, Wen J (2019) An efficient stacking model of multi-label classification based on pareto optimum. IEEE Access 7:127427–37

    Article  Google Scholar 

  57. Wu B, Zhong E, Horner A, Yang Q (2014) Music emotion recognition by multi-label multi-layer multi-instance multi-view learning. Proceedings of the Twenty-Second ACM international conference on Multimedia, pp 117–126

  58. Yang S, Hou C, Nie F, Wu Y (2012) Unsupervised maximum margin feature selection via l2,1,-norm minimization. Neural Comput Appl 21:1791–1799

    Article  Google Scholar 

  59. Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) L2,1-norm regularized discriminative feature selection for unsupervised. Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, pp 1589–1594

  60. Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19(10):2761–2773

    Article  MathSciNet  MATH  Google Scholar 

  61. Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Pattern Recogn 95:136–150

    Article  Google Scholar 

  62. Zhang M -L, Zhou Z -H (2007) ML-KNN: A lazy learning approach to multi-label learning. Pattern Recogn 40:2038–2048

    Article  MATH  Google Scholar 

  63. Zhang M -L, Zhou Z -H (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  64. Zhang M L, Wu L (2014) LIFT: Multi-Label learning with Label-Specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120

    Article  Google Scholar 

  65. Zhang P, Liu G, Gao W (2019) Distinguishing two types of labels for multi-label feature selection. Pattern Recogn 95:72–82

    Article  Google Scholar 

  66. Zhang Y, Gong D, Sun X, Guo Y (2017) A PSO-based multi-objective multi-label feature selection method in classification. Sci Rep 7(1):1–12

    Google Scholar 

  67. Zhang Y, Wu J, Cai Z, Philip S Y (2020) Multi-view Multi-label Learning with Sparse Feature Selection for Image Annotation. IEEE Trans Multimed 22(11):2844–2857

  68. Zhou H, Zhang Y, Zhang Y, Liu H (2019) Feature selection based on conditional mutual information: minimum conditional relevance and minimum conditional redundancy. Appl Intell 49:883–896

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by grants from the National Natural Science Foundation of China (No. 42076058), the Fujian Provincial Science and Technology Major Project (No. 2020HZ02014), Scientific Research Funds of Huaqiao University (NO. 605-50Y21005).

Author information

Authors and Affiliations

  1. Department of Automation, Xiamen University, Xiamen, 361005, China

    Yuling Fan & Shunxiang Wu

  2. Department of Computer Science and Technology, Huaqiao University, Xiamen, 361021, China

    Jinghua Liu

Authors
  1. Yuling Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinghua Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shunxiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shunxiang Wu.

Ethics declarations

Conflict of Interests

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, Y., Liu, J. & Wu, S. Exploring instance correlations with local discriminant model for multi-label feature selection. Appl Intell 52, 8302–8320 (2022). https://doi.org/10.1007/s10489-021-02799-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02799-0

Keywords

Search

Navigation