Sparse and low-rank representation for multi-label classification | Applied Intelligence Skip to main content
Springer Nature Link
Log in

Sparse and low-rank representation for multi-label classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label learning deals with the problem where each instance may be associated with multiple labels simultaneously, and how to discover and exploit the label correlations is one of important research issues. In this paper, we propose a novel sparse and low-rank representation-based method for multi-label classification (SLMLC), which can automatically exploit the asymmetric correlations among labels while learning the model parameters in a unified learning framework. More specifically, we assume that the weight matrix is divided into a sparse matrix and a low-rank matrix, where the sparse and low-rank matrices are utilized to capture the specific features that are relevant to each label and the shared feature subspace among all labels, respectively. Then, we integrate multi-label classification and label correlations into a joint learning framework to learn the correlations among labels and the model parameters simultaneously. Lastly, the formulation is transformed into its convex surrogate due to its non-convexity, and we solve it by developing an alternating iterative method. Experimental results on fifteen data sets in terms of six evaluation criteria show that SLMLC achieves superior performance compared to the state-of-the-art multi-label classification algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26 (8):1819–1837

    Article  Google Scholar 

  2. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39 (2/3):135–168

    Article  MATH  Google Scholar 

  3. Zhang ML, Zhou ZH (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  4. Yan Y, Wang Y, et al., Gao WC (2018) LSTM: Multi-label ranking for document classification. Neural Process Lett 47(1)):117–138

  5. Boutell MR, Luo J, Luo JB, Shen XP, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771

    Article  Google Scholar 

  6. Wang C, Yan S, Zhang L, Zhang HJ (2009) Multi-label sparse coding for automatic image annotation. In: IEEE conference on computer vision and pattern recognition, Miami, pp 1643– 1650

  7. Wang J, Yang Y, Mao JH et al (2016) Cnn-rnn: a unified framework for multi-label image classification. In: IEEE conference on computer vision and pattern recognition, pp 2285–2294

  8. Tan QY, et al., Liu YZ, Chen X (2017) Multi-Label classification based on low rank representation for image annotation. Remote Sens 9(2):109

  9. Qi GJ, Hua XS, Rui Y, et al. (2007) Correlative multi-label video annotation. In: Proceedings of the 15th ACM international conference on multimedia, Augsburg, pp 17–26

  10. Janwe NJ, Bhoyar KK (2018) Multi-label semantic concept detection in videos using fusion of asymmetrically trained deep convolutional neural networks and foreground driven concept co-occurrence matrix. Appl Intell 48:2047–2066

    Article  Google Scholar 

  11. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multilabel classification of music into emotions. In: Proceedings of the 9th international conference on music information retrieval, Philadephia, pp 325–330

  12. Zhang ML, Zhang K (2010) Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining, Washington, pp 999–1008

  13. Gu Q, Li Z, Han J (2011) Correlated multi-label feature selection. In: Proceedings of the 20th ACM international conference on information and knowledge management, Glasgow, pp 1087–1096

  14. Zhang Y, Yeung DY (2013) Multilabel relationship learning. ACM Trans Knowl Discov Data 7(2):1–30

    Article  Google Scholar 

  15. Bi W, James TK (2014) Multilabel classification with label correlations and missing labels. In: Proceedings of the Twenty-Eighth AAAI conference on artificial intelligence, Quebec, pp 1680–1686

  16. Huang SJ, Yu Y, Zhou ZH (2012) Multi-label hypothesis reuse. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, Beijing, pp 525–533

  17. He ZF, Yang M, Liu HD (2014) Joint learning of multi-label classification and label correlations. J Soft 25(9):1967–1981. (in Chinese)

    MATH  Google Scholar 

  18. Xu LL, et al., Wang Z, Shen ZF (2014) Learning low-rank label correlations for multi-label classification with missing labels. In: IEEE international conference on data mining (ICDM), Shenzhen, pp 1067–1072

  19. He ZF, Yang M, Liu HD (2015) Multi-task joint feature selection for multi-label classification. Chin J Electron 24(CJE-2):281–287

    Article  Google Scholar 

  20. Elisseeff A, Weston J (2001) A kernel method for multi-labelled classification. In: Proceedings of the 14th conference on neural information processing systems (NIPS2001), Vancouver, pp 681–687

  21. Zhang ML, Pena JM, Robles V (2009) Feature selection for multi-label naive Bayes classification. Inf Sci 179(19):3218–3229

    Article  MATH  Google Scholar 

  22. Hullermeier E, Furnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16):1897–1916

    Article  MathSciNet  MATH  Google Scholar 

  23. Furnkranz J, Hullermeier E, et al. (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153

    Article  Google Scholar 

  24. Read J, Pfahringer B, Holmes G, et al. (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359

    Article  MathSciNet  Google Scholar 

  25. Tsoumakas G, Katakis I, Vlahavas I (2008) Effective and efficient multilabel classification in domains with large number of labels. In: Proceedings of ECML/PKDD 2008 workshop on mining multidimensional data, Antwerp, pp 30–44

  26. Tsoumakas G, Katakis I, Vlahavas I (2011) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079–1089

    Article  Google Scholar 

  27. Godbole S, Sarawagi S (2004) Discriminative methods for multi-labeled classification. In: Dai H, Srikant R, Zhang C (eds) Lecture notes in artificial intelligence 3056. Springer, Berlin, pp 22–30

  28. Ghamrawi N, Mccallum A (2005) Collective multilabel classification. In: Proceedings of the 14th ACM international conference on information and knowledge management, Bremen, pp 195– 200

  29. Chen G, Song YQ, Wang F, et al. (2008) Semi-supervised multi-label learning by solving a sylvester equation. In: SIAM conference on data mining, Atlanta, pp 410–419

  30. Weng W, Lin YJ, Wu SX, et al. (2018) Multi-label learning based on label-specific features and local pairwise label correlation. Neurocomputing 273:385–394

    Article  Google Scholar 

  31. Zhu Y, Kwok JT, Zhou ZH (2017) Multi-label learning with global and local label correlation. IEEE Transactions on Knowledge and Data Engineering. arXiv:1704.01415

  32. Guo Y, Gu SC (2011) Multi-label classification using conditional dependency networks. In: Walsh T (ed) Proceedings of the 22nd international joint conference on artificial intelligence, AAAI Press, pp 1300–1305

  33. Guo Y, Xue W (2013) Probabilistic multi-label classification with sparse feature learning. In: Rossi F (ed) Proceedings of the 23rd international joint conference on artificial intelligence, AAAI Press, pp 1373–1379

  34. Zhang ML (2011) LIFT: multi-label learning with label-specific features. In: Walsh T (ed) Proceedings of the 22nd international joint conference on artificial intelligence, Barcelona, pp 1609–1614

  35. Ji S, Ye J (2009) Linear dimensionality reduction for multi-label classification. In: Proceedings of the 21st international joint conference on artificial intelligence, California, pp 1077–1082

  36. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    Article  MATH  Google Scholar 

  37. Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings of New Zealand computer science research student conference, Christchurch, pp 143-150

  38. Ji S, Tang L, Yu S, Ye J (2010) A shared-subspace learning framework for multi-label classification. ACM Trans Knowl Discovery Data 4(2):1–29

    Article  Google Scholar 

  39. Chen J, Liu J, Ye J (2012) Learning incoherent sparse and low-rank patterns from multiple tasks. ACM Trans Knowl Discovery Data 5(4):1–31

    Article  Google Scholar 

  40. Gutta S, Cheng Q (2013) Joint multitask feature learning and classifier design. In: Proceedings of the 47th annual conference on information sciences and systems, pp 1–5

  41. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  42. Fazel M, Hindi H, Boyd S (2001) A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the 2001 American control conference, pp 4734–4739

  43. Ji S, Ye J (2009) An accelerated gradient method for trace normminimization. In: Proceedings of the 26th annual international conference on machine learning, Montreal, pp 457–464

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China under Grants 61876087, 61502058, 61432008, the Science and Technology Research Project of Jiangxi Provincial Education Department under Grant GJJ151262, Natural Science Foundation of Educational Committee of Jiangsu Province under Grant 15KJB520002, and the Social Science Research Project of Pingxiang under Grant 2017XW02. The authors would like to thank the anonymous reviewers and the editors for their constructive and valuable comments.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Nanjing Normal University, No. 1, Wenyuan Road, Qixia District, Nanjing, Jiangsu, 210023, People’s Republic of China

    Zhi-Fen He & Ming Yang

  2. School of Information and Computer Engineering, Pingxiang University, No.211, North Pingan Avenue, Anyuan District, Pingxiang, Jiangxi, 337055, People’s Republic of China

    Zhi-Fen He

Authors
  1. Zhi-Fen He
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ming Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ming Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, ZF., Yang, M. Sparse and low-rank representation for multi-label classification. Appl Intell 49, 1708–1723 (2019). https://doi.org/10.1007/s10489-018-1345-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1345-5

Keywords