Superpixel attention guided network for accurate and real-time salient object detection | Multimedia Tools and Applications Skip to main content

Advertisement

Springer Nature Link
Log in

Superpixel attention guided network for accurate and real-time salient object detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Edge information has been proven to be effective for remedying the unclear boundaries of salient objects. Current salient object detection (SOD) methods usually utilize edge detection as an auxiliary task to introduce explicit edge information. However, edge detection is unable to provide the indispensable regional information for SOD, which may result in incomplete salient objects. To alleviate this risk, observing that superpixels hold the inherent property that contains both edge and regional information, we propose a superpixel attention guided network (SAGN) in this paper. Specifically, we first devise a novel supervised deep superpixel clustering (DSC) method to form the relation between superpixels and SOD. Based on the DSC, we build a superpixel attention module (SAM), which provides superpixel attention maps that can neatly separate different salient foreground and background regions, while preserving accurate boundaries of salient objects. Under the guidance of the SAM, a lightweight decoder with a simple but effective structure is able to yield high-quality salient objects with accurate and sharp boundaries. Hence, our model only contains less than 5 million parameters and achieves a real-time speed of around 40 FPS. Whilst offering a lightweight model and fast speed, our method still outperforms other 11 state-of-the-art approaches on six benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 1597–1604

  2. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  3. Alpert S, Galun M, Brandt A, Basri R (2011) Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell 34(2):315–327

    Article  Google Scholar 

  4. Borji A, Frintrop S, Sihite D N, Itti L (2012) Adaptive object tracking by learning background context. In: 2012 IEEE Computer society conference on computer vision and pattern recognition workshops. IEEE, pp 23–30

  5. Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. In: Proceedings of the european conference on computer vision (ECCV), pp 132–149

  6. Chen C, Sun X, Hua Y, Dong J, Xv H (2020) Learning deep relations to promote saliency detection.. In: AAAI, pp 10510–10517

  7. Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587

  8. Chen S, Tan X, Wang B, Lu H, Hu X, Fu Y (2020) Reverse attention-based residual network for salient object detection. IEEE Trans Image Process 29:3763–3776

    Article  Google Scholar 

  9. Deng Z, Hu X, Zhu L, Xu X, Qin J, Han G, Heng P-A (2018) R3net: Recurrent residual refinement network for saliency detection. In: Proceedings of the 27th international joint conference on artificial intelligence. AAAI Press, pp 684–690

  10. Fang H, Gupta S, Iandola F, Srivastava R K, Deng L, Dollár P, Gao J, He X, Mitchell M, Platt J C et al (2015) From captions to visual concepts and back. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1473–1482

  11. Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1623–1632

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  13. He S, Lau Rynson WH, Liu W, Huang Z, Yang Q (2015) Supercnn: A superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115(3):330–344

    Article  MathSciNet  Google Scholar 

  14. Hou Q, Cheng M-M, Hu X, Borji A, Tu Z, Torr PHS (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212

  15. Hu P, Shuai B, Liu J, Wang G (2017) Deep level sets for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2300–2309

  16. Itti L, Koch C (2001) Computational modelling of visual attention. Nat Rev Neurosci 2(3):194–203

    Article  Google Scholar 

  17. Ji X, Henriques J F, Vedaldi A (2019) Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 9865–9874

  18. Kingma D P, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  19. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  20. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463

  21. Li G, Yu Y (2016) Deep contrast learning for salient object detection. In: Proceedings of the IEEE conference on computer vision and patern recognition, pp 478–487

  22. Li X, Yang F, Cheng H, Liu W, Shen D (2018) Contour knowledge transfer for salient object detection. In: Proceedings of the european conference on computer vision (ECCV), pp 355–370

  23. Li Y, Hou X, Koch C, Rehg J M, Yuille A L (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287

  24. Liu J-J, Hou Q, Cheng M-M, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3917–3926

  25. Luo Z, Mishra A, Achkar A, Eichel J, Li S, Jodoin P-M (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6609–6617

  26. Parimala M, Swarna Priya RM, Praveen Kumar Reddy M, Lal Chowdhary C, Kumar Poluru R, Khan S (2021) Spatiotemporal-based sentiment analysis on tweets for risk assessment of event using deep learning approach. Softw: Pract Exper 51(3):550–570

    Google Scholar 

  27. Reddy G T, Bhattacharya S, Ramakrishnan S S, Chowdhary C L, Hakak S, Kaluri R, Reddy M P K (2020) An ensemble based machine learning model for diabetic retinopathy classification. In: 2020 international conference on emerging trends in information technology and engineering (ic-ETITE). IEEE, pp 1–6

  28. Reddy T, RM S P, Parimala M, Chowdhary C L, Hakak S, Khan W Z et al (2020) A deep neural networks based model for uninterrupted marine environment monitoring. Comput Commun 157:64–75

    Article  Google Scholar 

  29. RM S P, Maddikunta P K R, Parimala M, Koppu S, Gadekallu T R, Chowdhary C L, Alazab M (2020) An effective feature engineering for dnn using hybrid pca-gwo for intrusion detection in iomt architecture. Comput Commun 160:139–149

    Article  Google Scholar 

  30. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  31. Schuurmans M, Berman M, Blaschko M B (2018) Efficient semantic image segmentation with superpixel pooling. arXiv:1806.02705

  32. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  33. Somayaji S R K, Alazab M, MK M, Bucchiarone A, Chowdhary C L, Gadekallu T R (2020) A framework for prediction and storage of battery life in iot devices using dnn and blockchain. arXiv:2011.01473

  34. Tang M, Gorelick L, Veksler O, Boykov Y (2013) Grabcut in one cut. In: Proceedings of the IEEE international conference on computer vision, pp 1769–1776

  35. Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern, pp 136–145

  36. Wang L, Wang L, Lu H, Zhang P, Ruan X (2016) Saliency detection with recurrent fully convolutional networks. In: European conference on computer vision. Springer, pp 825–841

  37. Wang T, Borji A, Zhang L, Zhang P, Lu H (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4019–4028

  38. Wang W, Shen J, Porikli F (2015) Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3395–3402

  39. Wang W, Zhao S, Shen J, Hoi SCH, Borji A (2019) Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1448–1457

  40. Wu R, Feng M, Guan W, Wang D, Lu H, Ding E (2019) A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8150–8159

  41. Wu Z, Su L, Huang Q (2019) Stacked cross refinement network for edge-aware salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 7264–7273

  42. Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International conference on machine learning, pp 478–487

  43. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500

  44. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  45. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1155–1162

  46. Yang B, Fu X, Sidiropoulos N D, Hong M (2017) Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In: international conference on machine learning. PMLR, pp 3861–3870

  47. Yang C, Zhang L, Lu H, Ruan X, Yang M-H (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  48. Yang F, Sun Q, Jin H, Zhou Z (2020) Superpixel segmentation with fully convolutional networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13964–13973

  49. Zeng Y, Zhang P, Zhang J, Lin Z, Lu H (2019) Towards high-resolution salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 7234–7243

  50. Zhang P, Wang D, Lu H, Wang H, Ruan X (2017) Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 202–211

  51. Zhang P, Wang D, Lu H, Wang H, Yin B (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE international conference on computer vision, pp 212–221

  52. Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 714–722

  53. Zhao J-X, Liu J-J, Fan D-P, Cao Y, Yang J, Cheng M-M (2019) Egnet: Edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788

  54. Zhao K, Gao S, Wang W, Cheng M-M (2019) Optimizing the f-measure for threshold-free salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8849–8857

  55. Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1265–1274

Download references

Acknowledgements

The work is supported by National Key R&D Program of China (2018YFC0309400), National Natural Science Foundation of China (61871188), Guangzhou city science and technology research projects (201902020008).

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, South China University of Technology, GuangZhou, China

    Zhiheng Zhou, Yongfan Guo, Junchu Huang, Ming Dai & Ming Deng

  2. Key Laboratory of Big Data and Intelligent Robot, South China University of Technology, Ministry of Education, GuangZhou, China

    Zhiheng Zhou, Yongfan Guo, Junchu Huang, Ming Dai & Ming Deng

  3. School of Digital Arts & Design, Dalian Neusoft University of Information, Dalian, China

    Qingjun Yu

Authors
  1. Zhiheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongfan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junchu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ming Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingjun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiheng Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Z., Guo, Y., Huang, J. et al. Superpixel attention guided network for accurate and real-time salient object detection. Multimed Tools Appl 81, 38921–38944 (2022). https://doi.org/10.1007/s11042-022-13083-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13083-9

Keywords

Search

Navigation