Semi-supervised multi-label feature learning via label enlarged discriminant analysis | Knowledge and Information Systems Skip to main content
Springer Nature Link
Log in

Semi-supervised multi-label feature learning via label enlarged discriminant analysis

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Multi-label learning arises frequently in various domains including computer vision and machine learning and has attracted great interest in the last decades. However, current multi-label classification methods may be deficient in many real applications with following two constraints: (1) lack of sufficient labeled data and (2) high dimensionality in feature space. To address these challenges, in this paper, we propose a new semi-supervised multi-label feature learning algorithm named as label enlarged discriminant analysis. Different from supervised multi-label learning methods, the proposed algorithm can utilize the information from both labeled data and unlabeled data in an effective way. The proposed algorithm enlarges the multi-label information from the labeled data to the unlabeled data through a special designed multi-label label propagation method. Thus, it can take both labeled and unlabeled data into consideration. It then learns a transformation matrix to perform feature learning to reduce the high dimensionality by incorporating the enlarged multi-label information. In this way, the proposed algorithm can preserve more discriminative information by utilizing both labeled and unlabeled data simultaneously. We have analyzed in theory and extensive experimental results are carried out upon several data sets. They all validate the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://sourceforge.net/projects/mulan/files/datasets/Emotions.

  2. http://cse.seu.edu.cn/people/zhangml.

  3. https://sourceforge.net/projects/mulan/files/datasets/Mediamill.

  4. https://sourceforge.net/projects/mulan/files/datasets/Medical.

  5. http://sourceforge.net/projects/mulan/files/datasets/Scene.

  6. https://sourceforge.net/projects/mulan/files/datasets/Tmc2007-500.

  7. https://sourceforge.net/projects/mulan/files/datasets/Bibtex.

  8. http://www.kecl.ntt.co.jp/as/members/ueda/yahoo.tar.gz.

  9. https://sourceforge.net/projects/mulan/files/datasets/Cbmi09-bow.

References

  1. Belkin M, Matveeva I, Niyogi P (2004) Regularization and semi-supervised learning on large graphs. In: COLT, pp 624–638

  2. Cai Z, Zhu W (2017) Multi-label feature selection via feature manifold learning and sparsity regularization. Int J Mach Learn Cybern 9(8):1321–1334. https://doi.org/10.1007/s13042-017-0647-y

    Article  Google Scholar 

  3. Chapelle O, Weston J, Schölkopf B (2003) Cluster kernels for semi-supervised learning. In: NIPS, pp 601–608

  4. Charte F, Rivera AJ, Jesus MJD, Herrera F (2014) Concurrence among imbalanced labels and its influence on multilabel resampling algorithms. In: HAIS, pp 110–121

  5. Chen G, Song Y, Wang F, Zhang C (2008) Semi-supervised multi-label learning by solving a sylvester equation. In: SDM, pp 410–419

  6. Fisher RA (1936) The use of multiple measures in taxonomic problems. Ann Hum Genet 7:179–188

    Google Scholar 

  7. Gibaja E, Ventura S (2015) A tutorial on multilabel learning. ACM Comput Surv 47(3):1–38

    Article  Google Scholar 

  8. Gretton A, Bousquet O, Smola A, Scholkopf B (2005) Measuring statistical dependence with Hilbert–Schmidt norms. In: ALT, vol 16. Springer, pp 63–78

  9. Guo B, Hou C, Nie F, Yi D (2016) Semi-supervised multi-label dimensionality reduction. In: ICDM

  10. Guo Y, Schuurmans D (2012) Semi-supervised multi-label classification. In: ECML/PKDD, pp 355–370

  11. He X, Niyogi P (2003) Locality preserving projections. In: NIPS, pp 153–160

  12. Herrera F, Charte F, Rivera AJ, Del Jesus MJ (2016) Multilabel classification: problem analysis, metrics and techniques. Springer, Berlin

    Book  Google Scholar 

  13. Horn RA, Johnson CR (2013) Matrix analysis. Cambridge University, New York

    MATH  Google Scholar 

  14. Huang J, Li G, Huang Q, Wu X (2017) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48:1–14

    Google Scholar 

  15. Jolliffe I (1986) Principal component analysis. Springer, Berlin

    Book  Google Scholar 

  16. Kang F, Jin R, Sukthankar R (2006) Correlated label propagation with application to multi-label learning. In: 2006 IEEE computer society conference on computer vision and pattern recognition, pp 1719–1726

  17. Kong X, Ng MK, Zhou ZH (2013) Transductive multi-label learning via label set propagation. IEEE Trans Knowl Data Eng 99:704–719

    Article  Google Scholar 

  18. Li Y, Guo L (2007) Least squares linear discriminant analysis. In: ICML, pp 1087–1093

  19. Liu W, Zhang M (2011) Transductive multi-label learning via label set propagation. J Mach Learn Res 25:197–212

    Google Scholar 

  20. Liu Y, Jin R, Yang L (2006) Semi-supervised multi-label learning by constrained non-negative matrix factorization. In: AAAI, pp 421–426

  21. Liu Y, Liu Y, Wang C, Wang X, Zhou P, Yu G, Chan KCC (2015) What strikes the strings of your heart? Multi-label dimensionality reduction for music emotion analysis via brain imaging. IEEE Trans Auton Mental Dev 7(3):176–188

    Article  Google Scholar 

  22. Nie F, Xiang S, Jia Y, Zhang C (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recognit 42:2615–2627

    Article  Google Scholar 

  23. Nie F, Xiang S, Liu Y, Zhang C (2010) A general graph-based semi-supervised learning with novel class discovery. Neural Comput Appl 19:549–555

    Article  Google Scholar 

  24. Nie F, Xu D, Li X, Xiang S (2011) Semi-supervised dimensionality reduction and classification through virtual label regression. IEEE Trans Syst Man Cybern Part B (Cybernet) 41:675–685

    Article  Google Scholar 

  25. Paige CC, Saunders MA (1982) Algorithm 583: LSQR: sparse linear equations and least squares problems. ACM Trans Math Softw 8(2):195–209

    Article  Google Scholar 

  26. Plaza A, Plaza J, Valencia D (2007) Impact of platform heterogeneity on the design of parallel algorithms for morphological processing of high-dimensional image data. J Supercomput 40:81–107

    Article  Google Scholar 

  27. Qian B, Davidson I (2010) Semi-supervised dimension reduction for multi-label classification. In: AAAI, pp 569–574

  28. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39:135–168

    Article  Google Scholar 

  29. Wang H, Chris D, Huang H (2010) Multi-label linear discriminant analysis. In: ECCV, pp 126–139

  30. Wang H, Huang H, Ding C (2009) Image annotation using multi-label correlated green’s function. In: ICCV, pp 197–212

  31. Wei Q, Yang Z, Junping Z, Wang Y (2009) Semi-supervised multi-label learning algorithm using dependency among labels. In: ICMLC, pp 112–116

  32. Wu L, Zhang ML (2013) Multi-label classification with unlabeled data: an inductive approach. ACML 29:197–212

    Google Scholar 

  33. Ye J (2007) Least squares linear discriminant analysis. In: ICML, pp 1087–1093

  34. Ye J, Ji S, Sun L (2011) Multi-label dimensionality reduction. Chapman and Hall, London

    Google Scholar 

  35. Yu D, Fu B, Xu G, Qin A (2018) Constrained nonnegative matrix factorization-based semi-supervised multilabel learning. Int J Mach Learn Cybern 10(5):1093–1100. https://doi.org/10.1007/s13042-018-0787-8

    Article  Google Scholar 

  36. Yu K, Yu S, Tresp V (2005) Multi-label informed latent semantic indexing. In: ACM SIGIR, pp 258–265

  37. Yu Y, Pedrycz W, Miao D (2014) Multi-label classification by exploiting label correlations. Expert Syst Appl Int J 41:2989–3004

    Article  Google Scholar 

  38. Yu Y, Wang J, Tan Q, Jia L, Yu G, Yu Y, Wang J, Tan Q, Jia L, Yu G (2017) Semi-supervised multi-label dimensionality reduction based on dependence maximization. IEEE Access 5(99):21927–21940

    Article  Google Scholar 

  39. Zhang CG, Zhang XH (2013) Graph-based semi-supervised multi-label learning method. In: MEC, pp 1021–1025

  40. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40:2038–2048

    Article  Google Scholar 

  41. Zhang Y, Zhou ZH (2010) Multilabel dimensionality reduction via dependence maximization. ACM Trans Knowl Discov Data 4:1503–1505

    Article  Google Scholar 

  42. Zhao M, Zhao Z, Zhang H (2013) A soft label based linear discriminant analysis for semi-supervised dimensionality reduction. In: IJCNN, pp 83–97

  43. Zhou D, Bousquet O, Lal TN (2004) Learning with local and global consistency. In: NIPS, pp 321–328

  44. Zhu X, Ghahraman Z, Lafferty J (2003) Semi-supervised learning using Gaussian fields and harmonic functions. In: ICML, pp 912–919

  45. Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers. https://doi.org/10.2200/S00196ED1V01Y200906AIM006

Download references

Acknowledgements

This work was supported by the NSF of China under Grant 61922087 and Grant 61906201, and the NSF for Distinguished Young Scholars of Hunan Province under Grant 2019JJ20020. Chenping Hou is the corresponding author.

Author information

Authors and Affiliations

  1. National University of Defense Technology, Changsha, 410073, China

    Baolin Guo, Hong Tao, Chenping Hou & Dongyun Yi

Authors
  1. Baolin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chenping Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongyun Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenping Hou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, B., Tao, H., Hou, C. et al. Semi-supervised multi-label feature learning via label enlarged discriminant analysis. Knowl Inf Syst 62, 2383–2417 (2020). https://doi.org/10.1007/s10115-019-01409-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-019-01409-3

Keywords

Search

Navigation