Single image super-resolution via global aware external attention and multi-scale residual channel attention network | International Journal of Machine Learning and Cybernetics Skip to main content
Springer Nature Link
Log in

Single image super-resolution via global aware external attention and multi-scale residual channel attention network

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Recently, deep convolutional neural networks (CNNs) have shown significant advantages in improving the performance of single image super-resolution (SISR). To build an efficient network, multi-scale convolution is commonly incorporated into CNN-based SISR methods via scale features with different perceptive fields. However, the feature correlations of the same sample are not fully utilized by the existing multi-scale SISR approaches, impeding the further improvement of reconstruction performance. In addition, the correlations between different samples are still left unexplored. To address these problems, this paper proposes a deep-connected multi-scale residual attention network (DMRAN) by virtue of the feature correlations of the same sample and the correlations between different samples. Specifically, we propose a deep-connected multi-scale residual attention block (DMRAB) to take fully advantage of the multi-scale and hierarchical features, which can effectively learn the local interdependencies between channels by adjusting the channel features adaptively. Meanwhile, a global aware external attention (GAEA) is introduced to boost the performance of SISR by learning the correlations between all the samples. Furthermore, we develop a deep feature extraction structure (DFES), which seamlessly combines the stacked deep-connected multi-scale residual attention groups (DMRAG) with GAEA to learn deep feature representations incrementally. Extensive experimental results on the public benchmark datasets show the superiority of our DMRAN to the state-of-the-art SISR methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The authors confirm that the data supporting the findings of this study are available within the article.

References

  1. Rasti P, Uiboupin T, Escalera S et al (2016) Convolutional neural network super resolution for face recognition in surveillance monitoring. In: Perales FJ, Kittler J (eds) International conference on articulated motion and deformable objects. Springer International Publishing, Cham, pp 175–184

    Chapter  Google Scholar 

  2. Oktay O, Bai W, Lee M et al (2016) Multi-input cardiac image super-resolution using convolutional neural networks. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W (eds) Medical image computing and computer-assisted intervention. Springer International Publishing, Cham, pp 246–254

    Google Scholar 

  3. Luo Y, Zhou L, Shu W et al (2017) Video satellite imagery super resolution via convolutional neural networks. IEEE Geosci Remote Sens Lett 14:2398–2402

    Article  Google Scholar 

  4. Keys RG (2003) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Proces 29:1153–1160

    Article  MathSciNet  Google Scholar 

  5. Romano Y, Protter, et al (2014) Single image interpolation via adaptive nonlocal sparsity-based modeling. IEEE Trans Image Process 23:3085–3098

    Article  MathSciNet  Google Scholar 

  6. Zhang M, Desrosiers C (2018) High-quality image restoration using low-rank patch regularization and global structure sparsity. IEEE Trans Image Process 28:868–879

    Article  MathSciNet  Google Scholar 

  7. Ren C, He X, Pu Y et al (2019) Enhanced non-local total variation model and multi-directional feature prediction prior for single image super resolution. IEEE Trans Image Process 28:3778–3793

    Article  MathSciNet  Google Scholar 

  8. Kim JH, Lee JS (2018) Deep residual network with enhanced upscaling module for super-resolution. IEEE/CVF Conf Comput Vis Patt Recogn Workshops. https://doi.org/10.1049/ell2.12689

    Article  Google Scholar 

  9. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. IEEE Conf Comput Vis Patt Recogn. https://doi.org/10.1109/CVPR.2016.182

    Article  Google Scholar 

  10. Dong C, Loy CC, He K et al (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38:295–307

    Article  Google Scholar 

  11. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. IEEE Comput Soc Conf Comput Vis Patt Recogn 34:275–282

    Google Scholar 

  12. Qin J, Huang Y, Wen W (2020) Multi-scale feature fusion residual network for single image super-resolution. Neurocomputing 379:334–342

    Article  Google Scholar 

  13. Li J, Fang F, Mei K et al (2018) Multi-scale residual network for image super-resolution. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) European conference on computer vision. Springer International Publishing, Cham, pp 527–542

    Google Scholar 

  14. Zhang Y, Li K, Li K et al (2018) Image super-resolution using very deep residual channel attention networks. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) European conference on computer vision. Springer International Publishing, Cham, pp 294–310

    Google Scholar 

  15. Zhang Y, Tian Y, Kong Y et al (2018) Residual dense network for image super-resolution. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) IEEE/CVF Conference on Computer Vision and Pattern Recognition. Springer International Publishing, Cham, pp 2472–2481

    Google Scholar 

  16. Ying T, Jian Y, Liu X (2017) “Image Super-Resolution via Deep Recursive Residual Network,” in IEEE Conference on Computer Vision & Pattern Recognition., , pp. 2790–2798

  17. Kim J, Lee J K, Lee K M (2016) “Deeply-Recursive Convolutional Network for Image Super-Resolution,” in IEEE Conference on Computer Vision and Pattern Recognition., , pp.1637–1645

  18. Chao D, Chen CL, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Leibe B, Matas J, Sebe N, Welling M (eds) European conference on computer vision. Springer International Publishing, Cham, pp 391–407

    Google Scholar 

  19. He K, Zhang X, Ren S, et al (2016) “Deep Residual Learning for Image Recognition,” in IEEE Conference on Computer Vision and Pattern Recognition, pp.770–778

  20. Lim B, Son S, Kim H, et al (2017) “Enhanced Deep Residual Networks for Single Image Super-Resolution,” in IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE, pp.1132–1140

  21. Huang G, Liu Z, Laurens V, et al (2016) “Densely Connected Convolutional Networks,” in IEEE Conference on Computer Vision and Pattern Recognition., , pp. 2261–2269

  22. Tai Y, Yang J, Liu X, et al (2017) “MemNet: A Persistent Memory Network for Image Restoration,” in IEEE International Conference on Computer Vision., , pp. 4549–4557

  23. Tong T, Li G, Liu X, et al (2017) “Image Super-Resolution Using Dense Skip Connections,” in IEEE International Conference on Computer Vision., , pp. 4809–4817

  24. Guo, M.H., et al (2021) “Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks,” CoRR, vol.abs/2105.02358

  25. Shi W, Caballero J, F Huszár, et al (2016)“Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network,” in IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883

  26. Lai W S, Huang J B, Ahuja N, et al (2017) “Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution,” in IEEE Conference on Computer Vision & Pattern Recognition., , pp.5835–5843

  27. Xiao M, Chuhua S, Yubin Y (2016) “Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections,” CoRR, vol.abs/1606.08921

  28. Szegedy C, Liu W, Jia Y, et al (2014) “Going Deeper with Convolutions,” in IEEE Computer Society., 2014, pp. 1–9

  29. Liu Y, Zhang X, Wang S, et al (2020) “Progressive Multi-Scale Residual Network for Single Image Super-Resolution,” CoRR, vol.abs/2007.09552

  30. Xiong C, Shi X, Gao Z et al (2020) Attention augmented multi-scale network for single image super-resolution. Appl Intell 51:935–951

    Article  Google Scholar 

  31. Jie H, Li S, Gang S, et al (2017) “Squeeze-and-Excitation Networks,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 7132–7141

  32. Dai T, Cai J, Zhang Y, et al (2019) “Second-order Attention Network for Single Image Super-Resolution,” in IEEE Conference on Computer Vision and Pattern Recognition., pp. 11065–11074

  33. Wang X, Girshick R, Gupta A, et al (2017) “Non-local Neural Networks,” CoRR, vol.abs/1711.07971

  34. Zhang Y, K Li, K Li, et al (2019) “Residual non-local attention networks for image restoration,” CoRR, vol.abs/1903.10082

  35. Mei Y, Fan Y, Zhou Y, et al (2020) “Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining,” In IEEE/CVF conference on computer vision and pattern recognition., , pp. 5689–5698

  36. Liu Z, Huang J, Zhu C et al (2021) Residual attention network using multi-channel dense connections for image super-resolution. Appl Intell 51:85–99

    Article  Google Scholar 

  37. Hu X, Mu H, Zhang X, et al (2020) “Meta-SR: A magnification-arbitrary network for super-resolution,” In: IEEE conference on computer vision and pattern recognition., 2020, pp. 1575–1584

  38. Haris M, Shakhnarovich G, Ukita N (2018) “Deep back-projection networks for super-resolution,” arXiv. arXiv, , pp. 1664–1673

  39. Sajjadi M, Scholkopf B, Hirsch M (2017) “EnhanceNet: single image super-resolution through automated texture synthesis,” In: IEEE International Conference on Computer Vision., , pp. 4501–4510

  40. Ledig C, Theis L, F Huszar, et al (2016) “Photo-realistic single image super-resolution using a generative adversarial network,” In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 105–114

  41. Wang Q, et al (2020) “ECA-Net: efficient channel attention for deep convolutional neural networks,” In: IEEE conference on computer vision and pattern recognition., 2020, pp. 11531–11539

  42. Agustsson E, Timofte R (2017) “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.1122–1131

  43. Kingma D, Ba J (2015) “Adam: a method for stochastic optimization,” in international conference on learning representations

  44. Bevilacqua M, Roumy A, Guillemot C, et al (2012) “Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization,” in IEEE International Conference on Acoustics, pp.1289–1292

  45. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: Boissonnat J-D, Chenin P, Cohen A, Gout C, Lyche T, Mazure M-L, Schumaker L (eds) International conference on curves and surfaces. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 711–730

    Google Scholar 

  46. Martin D, Fowlkes C, Tal D, et al (2002) “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” In: IEEE International Conference on Computer Vision, pp. 416–425

  47. Huang J B, Singh A, Ahuja N (2015) “Single image super-resolution from transformed self-exemplars,” In: IEEE Conference on Computer Vision and Pattern Recognition., pp. 5197–5206

  48. Matsui Y, Ito K, Aramaki Y et al (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76:21811–21838

    Article  Google Scholar 

  49. Fang F, Li J, Zeng T (2020) Soft-edge assisted network for single image super-resolution. IEEE Trans Image Process 29:4656–4668

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants 61801198 and 62276266.

Author information

Authors and Affiliations

  1. School of Intelligent Manufacturing, Jiangsu Vocational Institute of Architectural Technology, Xuzhou, 221000, Jiangsu Province, China

    Mingming Liu

  2. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, Jiangsu Province, China

    Mingming Liu, Sui Li, Bing Liu, Yuxin Yang & Chen Zhang

  3. Mine Digitization Engineering Research Center of Ministry of Education of the People’s Republic of China, Xuzhou, China

    Sui Li, Bing Liu, Yuxin Yang & Chen Zhang

  4. National Joint Engineering Laboratory of Internet Applied Technology of Mines, Xuzhou, 221008, Jiangsu Province, China

    Peng Liu

Authors
  1. Mingming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuxin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Bing Liu or Peng Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Mingming Liu and Sui Li are co-first authors.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, M., Li, S., Liu, B. et al. Single image super-resolution via global aware external attention and multi-scale residual channel attention network. Int. J. Mach. Learn. & Cyber. 15, 2309–2321 (2024). https://doi.org/10.1007/s13042-023-02030-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-02030-1

Keywords

Search

Navigation