From local to global: a multi-group feature enhancement network for non-uniform and dense haze removal | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

From local to global: a multi-group feature enhancement network for non-uniform and dense haze removal

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the past few years, significant research on single-image dehazing has developed rapidly. Despite this effort, it is still hard to remove the dense haze completely, particularly in complex real-world cases. The real-world haze is non-uniform and varied (light or dense). In the non-uniform case, the structure of the image can be destroyed. Besides, the procedure of dense haze removal usually leads to color distortion, detail loss and structure blurring, which increases the difficulty of image restoration. To solve these problems, we propose a multi-group feature enhancement network (MGFEN) based on a global and local context fusion pattern to remove haze progressively. Unlike previous methods, we develop a global feature fusion (GFF) module which takes a more global perspective to extract features and performs attention fusion with high-frequency features obtained from the Laplace pyramid to effectively preserve structure information of the image and remove artifacts caused by non-uniform haze. We also design a feature residual enhancement (FRE) module to improve image details and boost color fidelity by enhancing effective residuals group by group. The Experimental results of different datasets show that our MGFEN establishes the new state-of-the-art performance for real-world non-uniform and dense haze removal both in objective metrics and visual quality with better structure and color recovery ability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Ancuti CO, Ancuti C, Sbert M et al (2019) Dense-haze: a benchmark for image dehazing with dense-haze and haze-free images [C]//2019 IEEE international conference on image processing (ICIP). IEEE 1014–1018

  2. Ancuti CO, Ancuti C, Timofte R et al (2018) O-haze: a dehazing benchmark with real hazy and haze-free outdoor images [C]// Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 754–762

  3. Berman D, Avidan S (2016) Non-local image dehazing [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 1674–1682

  4. Berman D, Treibitz T, Avidan S (2018) Single image dehazing using haze-lines [J]. IEEE Trans Pattern Anal Mach Intell 42(3):720–734

    Article  Google Scholar 

  5. Bu Q, Luo J, Ma K, Feng H, Feng J (2020) An enhanced pix2pix dehazing network with guided filter layer [J]. Appl Sci 10(17):5898

    Article  Google Scholar 

  6. Chen Y, Li W, Sakaridis C et al (2018) Domain adaptive faster r-cnn for object detection in the wild [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 3339–3348

  7. Chen Z, Wang Y, Yang Y et al (2021) PSD: Principled synthetic-to-real dehazing guided by physical priors [C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7180–7189

  8. Dong H, Pan J, Xiang L et al (2020) Multi-scale boosted dehazing network with dense feature fusion [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2157–2167

  9. Gao SH, Cheng MM, Zhao K et al (2019) Res2net: A new multi-scale backbone architecture [J]. IEEE Trans Pattern Anal Mach Intell 43(2):652–662 Y-net-26

  10. Girshick R (2015) Fast r-cnn [C]// Proceedings of the IEEE international conference on computer vision. 1440–1448

  11. Guo CL, Yan Q, Anwar S et al (2022) Image Dehazing Transformer with Transmission-Aware 3D Position Embedding [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5812–5820

  12. Hassan H, Mishra P, Ahmad M, Bashir AK, Huang B, Luo B (2022) Effects of haze and dehazing on deep learning-based vision models [J]. Appl Intell 52:1–19

    Article  Google Scholar 

  13. He K, Sun J, Tang X (2010) Single image haze removal using dark channel prior [J]. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Google Scholar 

  14. Hong M, Xie Y, Li C et al (2020) Distilling image dehazing with heterogeneous task imitation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3462–3471

  15. Huang Y, Chen X (2021) Single remote sensing image dehazing using a dual-step cascaded residual dense network [C]//2021 IEEE international conference on image processing (ICIP). IEEE 3852–3856

  16. Kan S, Zhang Y, Zhang F et al (2022) A GAN-based input-size flexibility model for single image dehazing [J]. Signal Process Image Commun 102:116599

    Article  Google Scholar 

  17. Khan S, Naseer M, Hayat M, Zamir SW, Khan FS, Shah M (2022) Transformers in vision: a survey [J]. ACM Comput Surv (CSUR) 54(10s):1–41

    Article  Google Scholar 

  18. Li B, Peng X, Wang Z et al (2017) Aod-net: All-in-one dehazing network [C]//Proceedings of the IEEE international conference on computer vision. 4770–4778

  19. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2018) Benchmarking single-image dehazing and beyond [J]. IEEE Trans Image Process 28(1):492–505

    Article  MathSciNet  MATH  Google Scholar 

  20. Lin TY, Dollár P, Girshick R et al (2017) Feature pyramid networks for object detection [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. 2117–2125

  21. Liu W, Anguelov D, Erhan D et al (2016) Ssd: single shot multibox detector [C]// European conference on computer vision. Springer, Cham, 21–37

  22. Liu S, Huang D, Wang Y (2019) Learning spatial fusion for single-shot object detection [J]. arXiv preprint arXiv:1911.09516

  23. Liu X, Ma Y, Shi Z et al (2019) Griddehazenet: Attention-based multi-scale network for image dehazing [C]// Proceedings of the IEEE/CVF international conference on computer vision. 7314–7323

  24. Liu S, Qi L, Qin H et al (2018) Path aggregation network for instance segmentation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. 8759–8768

  25. Liu H, Wu Z, Li L et al (2022) Towards Multi-Domain Single Image Dehazing via Test-Time Training [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5831–5840

  26. Middleton WEK (1952) Vision through the atmosphere [M]. University of Toronto Press

  27. Qin X, Wang Z, Bai Y et al (2020) FFA-Net: Feature fusion attention network for single image dehazing [C]// Proceedings of the AAAI Conference on Artificial Intelligence. 34(07):11908–11915

  28. Ren W, Ma L, Zhang J et al (2018) Gated fusion network for single image dehazing [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3253–3261

  29. Sakaridis C, Dai D, Hecker S et al (2018) Model adaptation with synthetic and real data for semantic dense foggy scene understanding [C]//Proceedings of the European Conference on Computer Vision (ECCV). 687–704

  30. Sakaridis C, Dai D, Van Gool L (2018) Semantic foggy scene understanding with synthetic data [J]. Int J Comput Vis 126(9):973–992

    Article  Google Scholar 

  31. Shao Y, Li L, Ren W et al (2020) Domain adaptation for image dehazing [C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2808–2817

  32. Shao X, Wei C, Shen Y, Wang Z (2020) Feature enhancement based on CycleGAN for nighttime vehicle detection [J]. IEEE Access 9:849–859

    Article  Google Scholar 

  33. Sharma T, Agrawal I, Verma NK (2020) CSIDNet: compact single image dehazing network for outdoor scene enhancement [J]. Multimed Tools Appl 79(41):30769–30784

    Article  Google Scholar 

  34. Song Y, He Z, Qian H et al (2022) Vision Transformers for Single Image Dehazing [J]. arXiv preprint arXiv:2204.03883

  35. Wang C, Fan W, Wu Y, Su Z (2020) Weakly supervised single image dehazing [J]. J Vis Commun Image Represent 72:102897

    Article  Google Scholar 

  36. Wang T, Zhao L, Huang P, Zhang X, Xu J (2021) Haze concentration adaptive network for image dehazing [J]. Neurocomputing 439:75–85

    Article  Google Scholar 

  37. Wu H, Qu Y, Lin S et al (2021) Contrastive learning for compact single image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10551–10560

  38. Yang Y, Wang C, Liu R et al (2022) Self-Augmented Unpaired Image Dehazing via Density and Depth Decomposition [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2037–2046

  39. Yang HH, Yang CHH, Tsai YCJ (2020) Y-net: multi-scale feature aggregation network with wavelet structure similarity loss function for single image dehazing [C]// ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE 2628–2632

  40. Yu Y, Liu H, Fu M et al (2021) A two-branch neural network for non-homogeneous dehazing via ensemble learning [C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 193–202

  41. Zamir SW, Arora A, Khan S et al (2022) Restormer: Efficient transformer for high-resolution image restoration [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5728–5739

  42. Zhang H, Cisse M, Dauphin YN et al (2018) mixup: Beyond Empirical Risk Minimization [C]// International Conference on Learning Representations

  43. Zhang X, Dong H, Pan J et al (2021) Learning to restore hazy video: A new real-world dataset and a new method [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9239–9248

  44. Zhao W, Zhao Y, Feng L, Tang J (2022) Attention optimized deep generative adversarial network for removing uneven dense haze [J]. Symmetry 14(1):1

    Article  Google Scholar 

  45. Zotti C, Luo Z, Humbert O et al (2017) GridNet with automatic shape prior registration for automatic MRI cardiac segmentation [C]//international workshop on statistical atlases and computational models of the heart. Springer, Cham, 73–81

Download references

Funding

This study was supported by the Joint Fund of Ministry of Education for Equipment Pre-research (Grant number 8091B0203) and National Science and Technology Innovation 2030 Major Program (Grant number 2022ZD0205000).

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing, 100044, China

    Xiaotao Shao, Yan Guo, Yan Shen, Manyi Qian & Zhongli Wang

Authors
  1. Xiaotao Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manyi Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhongli Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Xiaotao Shao; Methodology: Xiaotao Shao; Formal analysis and investigation: Yan Guo; Writing - original draft preparation: Yan Guo; Writing - review and editing: Yan Shen; Funding acquisition: Yan Shen; Resources: Manyi Qian; Supervision: Manyi Qian; Validation: Zhongli Wang; Visualization: Zhongli Wang.

Corresponding author

Correspondence to Yan Shen.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shao, X., Guo, Y., Shen, Y. et al. From local to global: a multi-group feature enhancement network for non-uniform and dense haze removal. Multimed Tools Appl 82, 27057–27073 (2023). https://doi.org/10.1007/s11042-023-14950-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14950-9

Keywords

Search

Navigation