A deep hourglass-structured fusion model for efficient single image dehazing | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

A deep hourglass-structured fusion model for efficient single image dehazing

  • 1204: Multimedia Technology for Security and Surveillance in Degraded Vision
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hazy images obstruct the visibility of image content, which can negatively affect vision-based decision-making in multimedia systems and applications. Recently, convolutional neural networks (CNN) are proven with great benefit to remove single image haze, which has aroused research attention. However, in practice, previous works fail to fully exploit multi-scale features and restore the faithful image details from the hazy inputs, resulting in sub-optimal performance. In this paper, we propose a novel and high-efficiency deep hourglass-structured fusion model to address this issue, which also indicates the applicability of the modified hourglass architecture to remove haze. Unlike the conventional multi-scale learning schemes, top-down and bottom-up feature fusions are repeated, so each of the coarse-to-fine scale representations receives data of parallel ones, which allows for more flexible information exchange and aggregation at various scales. To be specific, we develop residual dense module as the backbone unit, while introducing the channel-wise attention mechanism to further enhance the representation ability of the network. As proved by extensive assessments demonstrate, our designed model outclasses existing ones and achieves the advanced performance on benchmark datasets and real hazy images. We have released source codes on GitHub: https://github.com/cxtalk/Hourglass-DehazeNet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Berman D, Treibitz T, Avidan S (2016) Non-local image Dehazing. In: 2016 IEEE/CVF conference on computer vision and pattern recognition. pp 1674-1682. https://doi.org/10.1109/cvpr.2016.185

  2. Cai B, Xu X, Jia K, Qing C, Tao D (2016) DehazeNet: An end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198. https://doi.org/10.1109/TIP.2016.2598681

    Article  MathSciNet  MATH  Google Scholar 

  3. Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G (2019) Gated context aggregation network for image Dehazing and Deraining. In: 2019 IEEE winter conference on applications of computer vision, pp 1375-1383. https://doi.org/10.1109/wacv.2019.00151

  4. Chen X, Li YF, Dai LG, Kong CH (2021) Hybrid high-resolution learning for single remote sensing satellite image Dehazing. IEEE Geosci Remote Sens Lett 19:1–5. https://doi.org/10.1109/LGRS.2021.3072917

    Article  Google Scholar 

  5. Chen X, Huang YF, Xu L (2021) Multi-scale hourglass hierarchical fusion network for single image Deraining. In: 2021 IEEE/CVF conference on computer vision and pattern recognition workshops, pp 872-879

  6. Dong H, Pan J, Xiang L, Hu Z, Zhang X, Wang F, Yang M-H (2020) Multi-scale boosted dehazing network with dense feature fusion. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, pp 2154-2164. https://doi.org/10.1109/CVPR42600.2020.00223

  7. Fattal R (2008) Single image dehazing. ACM Trans Graph 27(3):1–9. https://doi.org/10.1145/1360612.1360671

    Article  Google Scholar 

  8. Fattal R (2014) Dehazing using color-lines. ACM Trans Graph 34(1):1–14. https://doi.org/10.1145/2651362

    Article  Google Scholar 

  9. Fourure D, Emonet R, Fromont E, Muselet D, Tremeau A, Wolf C (2017) Residual conv-deconv grid network for semantic segmentation arxiv: 1707.07958

  10. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353. https://doi.org/10.1109/TPAMI.2010.168

    Article  Google Scholar 

  11. Huang G, Chen D, Li T, Wu F, Van Der Maaten L, Weinberger K (2018) Multi-scale dense networks for resource efficient image classification arxiv: 1703.09844

  12. Jiang K, Wang Z, Yi P, Chen C, Huang B, Luo Y, Ma J, Jiang J (2020) Multi-scale progressive fusion network for single image Deraining. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, pp 8343–8352. https://doi.org/10.1109/CVPR42600.2020.00837

  13. Levin A, Lischinski D, Weiss Y (2008) A closed-form solution to natural image matting. IEEE Trans Pattern Anal Mach Intell 30(2):228–242. https://doi.org/10.1109/TPAMI.2007.1177

    Article  Google Scholar 

  14. Li B, Peng X, Wang Z, Xu J, Feng D (2017) AOD-Net: All-in-one dehazing network. In: 2017 IEEE international conference on computer vision (ICCV), Venice, Italy, pp. 4780–4788. https://doi.org/10.1109/ICCV.2017.511

  15. Li RD, Pan JS, Li ZC, Tang JH (2018) Single image Dehazing via conditional generative adversarial network. In: IEEE/CVF conference on computer vision and pattern recognition, New York, pp. 8202–8211. https://doi.org/10.1109/cvpr.2018.00856

  16. Li Y, Miao Q, Liu R, Song J, Quan Y, Huang Y (2018) A multi-scale fusion scheme based on haze-relevant features for single image dehazing. Neurocomputing 283:73–86. https://doi.org/10.1016/j.neucom.2017.12.046

    Article  Google Scholar 

  17. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2019) Benchmarking single image dehazing and beyond. IEEE Trans Image Process 28(1):492–505. https://doi.org/10.1109/TIP.2018.2867951

    Article  MathSciNet  MATH  Google Scholar 

  18. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE/CVF conference on computer vision and pattern recognition workshops. pp 1132-1140. https://doi.org/10.1109/cvprw.2017.151

  19. Liu X, Ma Y, Shi Z, Chen J (2019) GridDehazeNet: attention-based multi-scale network for image Dehazing. In: 2019 IEEE/CVF international conference on computer vision, pp 7313-7322. https://doi.org/10.1109/iccv.2019.00741

  20. Narasimhan SG, Nayar SK (2002) Vision and the atmosphere. Int J Comput Vis 48(3):233–254. https://doi.org/10.1023/a:1016328200723

    Article  MATH  Google Scholar 

  21. Nayar SK, Narasimhan SG (1999) Vision in bad weather. The Proceedings of the Seventh IEEE International Conference on Computer Vision 2:820–827

  22. Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: proceedings of the European conference on computer vision (ECCV), pp 483-499. https://doi.org/10.1007/978-3-319-46484-8_29

  23. Qin X, Wang Z, Bai Y, Xie X, Jia H (2020) FFA-net: feature fusion attention network for single image dehazing. In: AAAI, pp. 11908–11915

  24. Qu Y, Chen Y, Huang J, Xie Y, Soc IC (2019) Enhanced Pix2pix Dehazing network. In: 2019 IEEE/CVF conference on computer vision and pattern recognition, pp 8152-8160. https://doi.org/10.1109/cvpr.2019.00835

  25. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang MH (2016) Single image dehazing via multi-scale convolutional neural networks. European Conference Comput Vis 9906:154–169. https://doi.org/10.1007/978-3-319-46475-6-10

    Article  Google Scholar 

  26. Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang MH (2018) Gated fusion network for single image dehazing. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, USA, pp. 1–9. https://doi.org/10.1109/cvpr.2018.00343

  27. Shao Y, Li L, Ren W, Gao C, Sang N (2020) Domain adaptation for image dehazing. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, pp 2805-2814. https://doi.org/10.1109/cvpr42600.2020.00288

  28. Sharma T, Agrawal I, Verma NK (2020) CSIDNet: compact single image dehazing network for outdoor scene enhancement. Multimed Tools Appl 79(41–42):30769–30784. https://doi.org/10.1007/s11042-020-09496-z

    Article  Google Scholar 

  29. Shen J, Li Z, Yu L, Xia G-S, Yang W (2020) Implicit euler ODE networks for single-image dehazing. In: 2020 IEEE/CVF conference on computer vision and pattern recognition workshops, pp 877-886. https://doi.org/10.1109/cvprw50498.2020.00117

  30. Sun K, Xiao B, Liu D, Wang J, Soc IC (2019) Deep high-resolution representation learning for human pose estimation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition. pp 5686-5696. https://doi.org/10.1109/cvpr.2019.00584

  31. Tan RT (2008) Visibility in bad weather from a single image. In: 2008 IEEE/CVF conference on computer vision and pattern recognition, pp 2347-2354. https://doi.org/10.1109/cvpr.2008.4587643

  32. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/tip.2003.819861

    Article  Google Scholar 

  33. Wang C, Zhang M, Su Z, Yao G (2020) Densely connected multi-scale de-raining net. Multimed Tools Appl 79(27):19595–19614. https://doi.org/10.1007/s11042-020-08855-0

    Article  Google Scholar 

  34. Woo S, Park J, Lee J-Y, Kweon IS (2018) CBAM: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3-19. https://doi.org/10.1007/978-3-030-01234-2_1

  35. Yang Y, Zhang D, Huang S, Wu J (2019) Multilevel and multiscale network for single-image super-resolution. IEEE Signal Process Lett 26(12):1877–1881. https://doi.org/10.1109/lsp.2019.2952047

    Article  Google Scholar 

  36. Yeh C-H, Huang C-H, Kang L-W (2020) Multi-scale deep residual learning-based single image haze removal via image decomposition. IEEE Trans Image Process 29:3153–3167. https://doi.org/10.1109/tip.2019.2957929

    Article  MATH  Google Scholar 

  37. Zhang H, Patel VM (2018) Densely Connected Pyramid Dehazing Network. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, New York, pp 3194–3203. https://doi.org/10.1109/cvpr.2018.00337

  38. Zhang Y, Ding L, Sharma G (2017) HAZERD: AN OUTDOOR SCENE DATASET AND BENCHMARK FOR SINGLE IMAGE DEHAZING. In: 2017 24th IEEE international conference on image processing (ICIP). pp 3205-3209

  39. Zhang H, Sindagi V, Patel VM (2018) Multi-scale single image Dehazing using perceptual pyramid deep network. In: proceedings 2018 IEEE/CVF conference on computer vision and pattern recognition workshops, pp 1015-1024. https://doi.org/10.1109/cvprw.2018.00135

  40. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 2472-2481. https://doi.org/10.1109/cvpr.2018.00262

  41. Zhang X, Dong H, Hu Z, Lai W-S, Wang F, Yang M-H (2019) Gated fusion network for joint image deblurring and super-resolution arxiv: 1807.10806

  42. Zung J, Tartavull I, Lee K, Seung HS (2017) An Error Detection and Correction Framework for Connectomics arxiv: 1708.02599

Download references

Author information

Authors and Affiliations

  1. College of Electronic Information Engineering, Shenyang Aerospace University, Shenyang, 110135, China

    Yufeng Li, Xiang Chen, Caihua Kong, Longgang Dai & Yufeng Huang

Authors
  1. Yufeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Caihua Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Longgang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yufeng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiang Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Chen, X., Kong, C. et al. A deep hourglass-structured fusion model for efficient single image dehazing. Multimed Tools Appl 81, 35247–35260 (2022). https://doi.org/10.1007/s11042-022-12312-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12312-5

Keywords

Search

Navigation