MCGKT-Net: Multi-level Context Gating Knowledge Transfer Network for Single Image Deraining | SpringerLink
Skip to main content
Springer Nature Link
Log in

MCGKT-Net: Multi-level Context Gating Knowledge Transfer Network for Single Image Deraining

  • Conference paper
  • First Online:
Computer Vision – ACCV 2020 (ACCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12623))

Included in the following conference series:

Abstract

Rain streak removal in a single image is a very challenging task due to its ill-posed nature in essence. Recently, the end-to-end learning techniques with deep convolutional neural networks (DCNN) have made great progress in this task. However, the conventional DCNN-based deraining methods have struggled to exploit deeper and more complex network architectures for pursuing better performance. This study proposes a novel MCGKT-Net for boosting deraining performance, which is a naturally multi-scale learning framework being capable of exploring multi-scale attributes of rain streaks and different semantic structures of the clear images. In order to obtain high representative features inside MCGKT-Net, we explore internal knowledge transfer module using ConvLSTM unit for conducting interaction learning between different layers and investigate external knowledge transfer module for leveraging the knowledge already learned in other task domains. Furthermore, to dynamically select useful features in learning procedure, we propose a multi-scale context gating module in the MCGKT-Net using squeeze-and-excitation block. Experiments on three benchmark datasets: Rain100H, Rain100L, and Rain800, manifest impressive performance compared with state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Sultani, W., Chen, C., Shah, M.: Real-world anomaly detectionin surveillance videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6479–6488 (2018)

    Google Scholar 

  2. Barnum, P.C., Narasimhan, S., Kanade, T.: Analysis of rain and snow in frequency space. Int. J. Comput. Vis. 86(2), 256 (2010)

    Article  Google Scholar 

  3. Xu, J., et al.: Star: a structure and texture aware retinex model. IEEE Trans. Image Process. 29, 5022–5037 (2020)

    Article  Google Scholar 

  4. Kang, L.-W., Lin, C.-W., Fu, Y.-H.: Automatic single-image-base drain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2012)

    Article  MathSciNet  Google Scholar 

  5. Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2736–2744 (2016)

    Google Scholar 

  6. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    Google Scholar 

  7. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning, pp. 326–366. MIT Press, Cambridge (2016)

    Google Scholar 

  8. Zheng, Y., Yu, X., Liu, M., Zhang, S.: Residual multiscale based single image deraining. In: Conference on BMVC (2019)

    Google Scholar 

  9. Jiang, K., et al.: Multi-scale progressive fusion network for single image deraining. arXiv:2003.10985 (2020). http://arxiv.org/abs/2003.10985

  10. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  11. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  12. Du, Y., Xu, J., Zhen, X., Cheng, M., Shao, L.: Conditional variational image deraining. IEEE Trans. Image Process. 29, 6288–6301 (2020). https://doi.org/10.1109/TIP.2020.2990606

    Article  Google Scholar 

  13. Du, Y., Xu, J., Qiu, Q., Zhen, X., Zhang, L.: Variational image deraining. In: IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA, 2020, pp. 2395–2404 (2020). https://doi.org/10.1109/WACV45572.2020.9093393

  14. Wang, G., Sun, C., Sowmya, A.: ErlNet: entangled representation learning for single image deraining. In: IEEE International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  15. Kang, L.W., Lin, C.W., Fu, Y.H.: Automatic single image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2012)

    Article  MathSciNet  Google Scholar 

  16. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: IEEE International Conference on Computer Vision (ICCV), pp. 3397–3405 (2015)

    Google Scholar 

  17. Chen, Y., Hsu, C.: A Generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: IEEE International Conference on Computer Vision, Sydney, NSW 2013, pp. 1968–1975 (2013). https://doi.org/10.1109/ICCV.2013.247

  18. Chang, Y., Yan, L., Zhong, S.: Transformed low-rank model for line pattern noise removal. IEEE International Conference on Computer Vision (ICCV), pp. 1726–1734 (2017)

    Google Scholar 

  19. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26, 2944–2956 (2017)

    Article  MathSciNet  Google Scholar 

  20. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1715–1723 (2017)

    Google Scholar 

  21. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. arXiv preprint, arXiv:1701.05957 (2017)

  22. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeezeand-excitation context aggregation net for single image deraining. In: European Conference on Computer Vision, pp. 262–277 (2018)

    Google Scholar 

  23. Fan, Z., Wu, H., Fu, X., Huang, Y., Ding, X.: Residual-guide network for single image deraining. In: Proceedings of the ACM Multimedia Conference, pp. 1751–1759. ACM (2018)

    Google Scholar 

  24. Wei, W., Meng, D., Zhao, Q., Xu, Z., Wu, Y.: Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3877–3886 (2019)

    Google Scholar 

  25. Ren, D., Zuo, W., Zhang, D., Zhang, L., Yang, M.-H.: Simultaneous fidelity and regularization learning for image restoration. IEEE Trans. Pattern Anal. Mach. Intell. (2019)

    Google Scholar 

  26. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015)

    Google Scholar 

  27. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1357–1366 (2017)

    Google Scholar 

  28. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. arXiv e-prints, arXiv:1701.05957 (2017)

  29. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  30. Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 695–704 (2018)

    Google Scholar 

  31. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  32. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004 (2016)

  33. Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. arXiv preprint arXiv:1805.06173 (2018)

  34. Yasarla, R., Patel, V.M.: Uncertainty guided multi-scale residual learning-using a cycle spinning CNN for single image de-raining. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8405–8414 (2019)

    Google Scholar 

  35. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 262–277. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_16

    Chapter  Google Scholar 

  36. Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019)

    Google Scholar 

Download references

Acknowledgement

This research was supported in part by the Grant-in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant No. 20K11867.

Author information

Authors and Affiliations

  1. Graduate School of Science and Technology for Innovation, Yamaguchi University, 1677-1 Yoshida, Yamaguchi City, Yamaguchi, 753-8511, Japan

    Kohei Yamamichi & Xian-Hua Han

Authors
  1. Kohei Yamamichi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xian-Hua Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kohei Yamamichi .

Editor information

Editors and Affiliations

  1. Waseda University, Tokyo, Japan

    Hiroshi Ishikawa

  2. Institute of Automation of Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

  3. Czech Technical University in Prague, Prague, Czech Republic

    Tomas Pajdla

  4. University of Pennsylvania, Philadelphia, PA, USA

    Jianbo Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yamamichi, K., Han, XH. (2021). MCGKT-Net: Multi-level Context Gating Knowledge Transfer Network for Single Image Deraining. In: Ishikawa, H., Liu, CL., Pajdla, T., Shi, J. (eds) Computer Vision – ACCV 2020. ACCV 2020. Lecture Notes in Computer Science(), vol 12623. Springer, Cham. https://doi.org/10.1007/978-3-030-69532-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69532-3_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69531-6

  • Online ISBN: 978-3-030-69532-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics