Deep recurrent residual channel attention network for single image super-resolution | The Visual Computer Skip to main content

Advertisement

Springer Nature Link
Log in

Deep recurrent residual channel attention network for single image super-resolution

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The models based on convolutional neural network have achieved excellent results in image super-resolution by acquiring prior knowledge from a large number of images, but such models still have problems such as the features between layers in the depth network cannot be effectively fused, the number of parameters is too large, and cross-channel feature learning is impossible. Based on this, a deep recursive residual channel attention network (DRRCAN) model was proposed in this paper. To solve the problem that the information between different layers in the deep network cannot be fused effectively, this paper constructs a channel feature fusion module, which can effectively fuse the feature information of different layers. To solve the problem that the parameters increase sharply due to the increase of network depth, recursive blocks are adopted in this paper, which greatly reduces the number of parameters in the deep network. The channel attention is integrated to enable the model to learn features across channels. In addition, to avoid gradient explosion or disappearance, residual modules, long skip connections are introduced to improve the stability and generalization ability of the model. Extensive benchmark evaluations validate the superiority of the proposed DRRCAN model compared with existing algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

All in this paper are standard datasets that have been proven by a large number of papers and scholars.

References

  1. Bevilacqua, M., Roumy, A., Guillemot. C., et al.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)

  2. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. IEEE, pp I–I (2004)

  3. Chudasama, V., Upla, K., Raja, K., et al.: Compact and progressive network for enhanced single image super-resolution-compresrnet. Vis. Comput. 38(11), 3643–3665 (2022)

    Article  Google Scholar 

  4. Dong, C., Loy, C.C., He, K., et al.: Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, pp. 184–199. Springer (2014)

  5. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, pp. 391–407. Springer (2016)

  6. Dosovitskiy, A., Beyer, L., Kolesnikov. A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. (2020) arXiv:2010.11929

  7. Du, Q., Gu, W., Zhang, L., et al.: Attention-based LSTM-CNNS for time-series classification. In: Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 410–411 (2018)

  8. Gao, Y., Qi, Z., Zhao, D.: Edge-enhanced instance segmentation by grid regions of interest. Vis. Comput. 39(3), 1137–1148 (2023)

    Article  Google Scholar 

  9. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  10. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)

  11. Huang, Y., Wang, W., Wang, L.: Bidirectional recurrent convolutional networks for multi-frame super-resolution. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

  12. Jing, Y., Lin, L., Li, X., et al.: An attention mechanism based convolutional network for satellite precipitation downscaling over China. J. Hydrol. 613(128), 388 (2022)

    Google Scholar 

  13. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

  14. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1637–1645 (2016)

  15. Kirkland, E. J.: Bilinear interpolation. In: Advanced Computing in Electron Microscopy, pp. 261–263. Springer, Berlin (2010)

  16. Ledig, C., Theis, L., Huszár, F., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)

  17. Li, Z., Yang, J., Liu, Z., et al.: Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3867–3876 (2019)

  18. Liang, J., Zhou, T., Liu, D., et al.: Clustseg: Clustering for Universal Segmentation. (2023) arXiv:2305.02187

  19. Liu, B., Chen, J.: A super resolution algorithm based on attention mechanism and SRGAN network. IEEE Access 9, 139138–139145 (2021)

    Article  Google Scholar 

  20. Liu, D., Cui, Y., Tan, W., et al.: SG-Net: spatial granularity network for one-stage video instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9816–9825 (2021)

  21. Liu, D., Liang, J., Geng, T., et al.: Tripartite feature enhanced pyramid network for dense prediction. IEEE Trans. Image Process. (2023)

  22. Liu, T., Cai, Y., Zheng, J., et al.: Beacon: a boundary embedded attentional convolution network for point cloud instance segmentation. Vis. Comput. pp. 1–11 (2022)

  23. Ma, C., Rao, Y., Cheng, Y., et al.: Structure-preserving super resolution with gradient guidance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7769–7778 (2020)

  24. Martin, D., Fowlkes, C., Tal, D., et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. IEEE, pp. 416–423 (2001)

  25. Mei, Y., Fan, Y., Zhou, Y.: Image super-resolution with non-local sparse attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3517–3526 (2021)

  26. Mnih, V., Heess, N., Graves. A., et al.: Recurrent models of visual attention. In: Advances in Neural Information Processing Systems, vol. 27 (2014)

  27. Paszke, A., Gross, S., Chintala, S., et al.: Automatic differentiation in pytorch (2017)

  28. Raiko, T., Valpola, H., LeCun, Y.: Deep learning made easier by linear transformations in perceptrons. In: Artificial Intelligence and Statistics, PMLR, pp. 924–932 (2012)

  29. Srivastava, R.K., Greff, K., Schmidhuber, J.: Training very deep networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

  30. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3147–3155

  31. Tai, Y., Yang, J., Liu, X., et al.: Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4539–4547 (2017)

  32. Tian, C., Xu, Y., Zuo, W., et al.: Coarse-to-fine CNN for image super-resolution. IEEE Trans. Multim. 23, 1489–1502 (2020)

    Article  Google Scholar 

  33. Wang, F., Jiang, M., Qian, C., et al.: Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2017)

  34. Wang, W., Han, C., Zhou, T., et al.: Visual recognition with deep nearest centroids (2022). arXiv:2209.07383

  35. Wang, W., Liang, J., Liu, D.: Learning equivariant segmentation with instance-unique querying (2022). arXiv:2210.00911

  36. Wang, Z., Yang, Y., Wang, Z., et al.: Self-tuned deep super resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2015)

  37. Yang, J., Wright, J., Huang, T.S., et al.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  38. Yang, W., Feng, J., Yang, J., et al.: Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans. Image Process. 26(12), 5895–5907 (2017)

    Article  MathSciNet  Google Scholar 

  39. Yang, X., Zhu, Y., Guo, Y., et al.: An image super-resolution network based on multi-scale convolution fusion. Vis. Comput. pp. 1–11 (2021)

  40. Yu, J., Xiao, C., Su, K.: A method of gibbs artifact reduction for pocs super-resolution image reconstruction. In: 2006 8th International Conference on Signal Processing. IEEE (2006)

  41. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730. Springer, Berlin (2010)

  42. Zhang, W., Liu, Y., Dong, C., et al.: RankSRGAN: super resolution generative adversarial networks with learning to rank. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 7149–7166 (2021)

    Article  Google Scholar 

  43. Zhang, Y., Li, K., Li, K., et al.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)

  44. Zhou, D., Liu, Y., Li, X., et al.: Single-image super-resolution based on local biquadratic spline with edge constraints and adaptive optimization in transform domain. Vis. Comput. pp. 1–16 (2022)

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (62002200, 62202268, 62272281) and Shandong Provincial Natural Science Foundation (ZR2020QF012, ZR2021QF134, ZR2021MF068, ZR2021MF015, ZR2021MF107).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong Technology and Business University, Yantai, 264005, China

    Yepeng Liu, Dezhi Yang, Fan Zhang & Qingsong Xie

  2. Shandong Future Intelligent Financial Engineering Laboratory, Yantai, 264005, China

    Yepeng Liu, Fan Zhang, Qingsong Xie & Caiming Zhang

  3. School of Software, Shandong University, Jinan, 250101, China

    Caiming Zhang

  4. Digital Media Technology Key Lab of Shandong Province, Jinan, 250014, China

    Caiming Zhang

Authors
  1. Yepeng Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Dezhi Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Fan Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qingsong Xie
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Caiming Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yepeng Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Yang, D., Zhang, F. et al. Deep recurrent residual channel attention network for single image super-resolution. Vis Comput 40, 3441–3456 (2024). https://doi.org/10.1007/s00371-023-03044-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-03044-0

Keywords