Abstract
RGB-D saliency detection aims to segment eye-catching objects from images with the help of depth. Although many excellent methods raised, it is still difficult to locate salient objects accurately and efficiently, which lies in two challenges: (1) It is difficult to seamlessly and efficiently integrate cross-modal features from RGB-D inputs; (2) Low-quality depth maps have a serious negative impact on the final prediction results. The existing methods use two backbone networks to extract saliency features, which also introduce much redundancy. To address issues, we propose a simple and efficient deep feature refinement module to extract complementary depth features. We also design a depth correction module to filter out noisy depth input adaptively. Experiments with 13 recently proposed methods on 7 datasets demonstrate the effectiveness of the proposed approach both quantitatively and qualitatively, especially in efficiency and compactness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Boer, P.T.D., Kroese, D., Mannor, S., Rubinstein, R.: A tutorial on the cross-entropy method. Ann. Oper. Res. 134, 19–67 (2002)
Borji, A., Cheng, M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)
Chen, H., Li, Y.: Three-stream attention-aware network for rgb-d salient object detection. IEEE Trans. Image Process. 28(6), 2825–2835 (2019)
Chen, H., Li, Y.: Progressively complementarity-aware fusion network for rgb-d salient object detection, pp. 3051–3060 (06 2018)
Chen, H., Li, Y., Su, D.: Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for rgb-d salient object detection. Pattern Recogn. 86, 376–385 (2018)
Chen, S., Fu, Y.: Progressively guided alternate refinement network for RGB-D salient object detection. CoRR abs/2008.07064 (2020)
Chen, Z., Cong, R., Xu, Q., Huang, Q.: Dpanet: Depth potentiality-aware gated attention network for rgb-d salient object detection. IEEE Trans. Image Process. 1, 7012–7024 (2020)
Fan, D.P., Lin, Z., Zhang, Z., Zhu, M., Cheng, M.M.: Rethinking rgb-d salient object detection: models, data sets, and large-scale benchmarks. IEEE Trans. Neural Netw. Learn. Syst. 1, 2075–2089 (2020)
Fan, D.P., Gong, C., Cao, Y., Ren, B., Cheng, M.M., Borji, A.: Enhanced-alignment measure for binary foreground map evaluation (2018)
Fu, K., Fan, D.P., Ji, G.P., Zhao, Q.: Jl-dcf: Joint learning and densely-cooperative fusion framework for rgb-d salient object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3049–3059 (2020)
Hong, S., You, T., Kwak, S., Han, B.: Online tracking by learning discriminative saliency map with convolutional neural network (2015)
Ju, R., Ge, L., Geng, W., Ren, T., Wu, G.: Depth saliency based on anisotropic center-surround difference. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1115–1119 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Li, G., Liu, Z., Ling, H.: Icnet: Information conversion network for rgb-d based salient object detection. IEEE Trans. Image Process. 29, 4873–4884 (2020)
Li, G., Zhu, C.: A three-pathway psychobiological framework of salient object detection using stereoscopic technology. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 3008–3014 (2017)
Li, N., Ye, J., Ji, Y., Ling, H., Yu, J.: Saliency detection on light field. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 2806–2813. IEEE Computer Society, USA (2014)
Liu, J., Hou, Q., Cheng, M., Feng, J., Jiang, J.: A simple pooling-based design for real-time salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3912–3921 (2019)
Liu, N., Zhang, N., Han, J.: Learning selective self-mutual attention for rgb-d saliency detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13753–13762 (2020)
Máttyus, G., Luo, W., Urtasun, R.: Deeproadmapper: extracting road topology from aerial images. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3458–3466 (2017)
Niu, Y., Geng, Y., Li, X., Liu, F.: Leveraging stereopsis for saliency analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 454–461 (2012)
Pang, Y., Zhang, L., Zhao, X., Lu, H.: Hierarchical dynamic filtering network for rgb-d salient object detection (2020)
Piao, Y., Ji, W., Li, J., Zhang, M., Lu, H.: Depth-induced multi-scale recurrent attention network for saliency detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7253–7262 (2019)
Piao, Y., Rong, Z., Zhang, M., Ren, W., Lu, H.: A2dele: adaptive and attentive depth distiller for efficient rgb-d salient object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9057–9066 (2020)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (09 2014)
Su, J., Li, J., Zhang, Y., Xia, C., Tian, Y.: Selectivity or invariance: boundary-aware salient object detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3798–3807 (2019)
Wang, W., Shen, J., Yang, R., Porikli, F.: Saliency-aware video object segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 20–33 (2018)
Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: 2003 The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, vol. 2, pp. 1398–1402 (2003)
Zeng, Y., Zhuge, Y., Lu, H., Zhang, L., Qian, M., Yu, Y.: Multi-source weak supervision for saliency detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6067–6076 (2019)
Zhang, F., Du, B., Zhang, L.: Saliency-guided unsupervised feature learning for scene classification. IEEE Trans. Geosci. Remote Sens. 53(4), 2175–2184 (2015)
Zhang, J., et al.: Uc-net: Uncertainty inspired rgb-d saliency detection via conditional variational autoencoders. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8579–8588 (2020)
Zhang, M., Ren, W., Piao, Y., Rong, Z., Lu, H.: Select, supplement and focus for rgb-d saliency detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3469–3478 (2020)
Zhao, J., Cao, Y., Fan, D., Cheng, M., Li, X., Zhang, L.: Contrast prior and fluid pyramid integration for rgbd salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3922–3931 (2019)
Zhao, R., Ouyang, W., Wang, X.: Unsupervised salience learning for person re-identification. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3586–3593 (2013)
Zhao, T., Wu, X.: Pyramid feature attention network for saliency detection (2019)
Zhao, X., Pang, Y., Zhang, L., Lu, H., Zhang, L.: Suppress and balance: a simple gated network for salient object detection (2020)
Zhao, X., Zhang, L., Pang, Y., Lu, H., Zhang, L.: A single stream network for robust and real-time rgb-d salient object detection (2020)
Zhu, C., Cai, X., Huang, K., Li, T.H., Li, G.: Pdnet: prior-model guided depth-enhanced network for salient object detection. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 199–204 (2019)
Acknowledgement
Supported by the Natural Science Foundation of China (No. 61802336 No. 61806175 No. 62073322), Jiangsu Province 7th Projects for Summit Talents in Six Main Industries, Electronic Information Industry (DZXX-149, No.110), Yangzhou University “Qinglan Project”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, J., Yan, G., Xu, X., Wang, J., Chen, S., Hu, X. (2021). Efficient Depth-Included Residual Refinement Network for RGB-D Saliency Detection. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-87361-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87360-8
Online ISBN: 978-3-030-87361-5
eBook Packages: Computer ScienceComputer Science (R0)