{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T00:36:21Z","timestamp":1722990981056},"reference-count":54,"publisher":"Institution of Engineering and Technology (IET)","issue":"10","license":[{"start":{"date-parts":[[2024,5,13]],"date-time":"2024-05-13T00:00:00Z","timestamp":1715558400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62061049","12263008"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["ietresearch.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["IET Image Processing"],"published-print":{"date-parts":[[2024,8]]},"abstract":"Abstract<\/jats:title>Image denoising aims to remove noise from images and improve the quality of images. However, most image denoising methods heavily rely on pairwise training strategies and strict prior knowledge about image structure or noise distribution. While these methods exhibit significant results when handling known types of noise, their generalization performance diminishes when confronted with images containing unknown noise distributions. To address this issue, a two\u2010stage approach is introduced for enhancing the generalizability of image denoising. The proposed method does not rely on a large amount of paired data or prior knowledge of the noise type and level. Instead, it constructs a denoising pipeline with improved generalizability through an MLP\u2010based denoiser and generative diffusion prior. Specifically, in the first stage, an initial denoised image is predicted with a structure resembling that of the underlying clean image by introducing an MLP\u2010based U\u2010shaped denoising network aided by an implicit structural prior. In the second stage, the generalizability and quality of the denoiser are further enhanced by conditioning the result obtained from the previous stage on the pretrained denoising diffusion null\u2010space model. Extensive experimentation on multiple datasets demonstrates that this method exhibits better denoising performance and generalizability than other image denoising methods.<\/jats:p>","DOI":"10.1049\/ipr2.13122","type":"journal-article","created":{"date-parts":[[2024,5,13]],"date-time":"2024-05-13T09:07:51Z","timestamp":1715591271000},"page":"2625-2644","update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Improving the generalization of image denoising via structure\u2010preserved MLP\u2010based denoiser and generative diffusion prior"],"prefix":"10.1049","volume":"18","author":[{"given":"Jing","family":"Wu","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering Yunnan University Kunming China"}]},{"given":"Ruilin","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering Yunnan University Kunming China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-0721-9596","authenticated-orcid":false,"given":"Hao","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering Yunnan University Kunming China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-8449-6861","authenticated-orcid":false,"given":"Guowu","family":"Yuan","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering Yunnan University Kunming China"}]}],"member":"265","published-online":{"date-parts":[[2024,5,13]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2662206"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2839891"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.12.024"},{"key":"e_1_2_11_5_1","doi-asserted-by":"crossref","unstructured":"Liang T. Jin Y. Li Y. Wang T.:Edcnn: Edge enhancement\u2010based densely connected network with compound loss for low\u2010dose ct denoising. In:2020 15th IEEE International Conference on Signal Processing (ICSP) pp.193\u2013198. Beijing China (2020).https:\/\/doi.org\/10.1109\/ICSP48669.2020.9320928","DOI":"10.1109\/ICSP48669.2020.9320928"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3166956"},{"key":"e_1_2_11_7_1","unstructured":"Reis D. Kupec J. Hong J. Daoudi A.:Real\u2010time flying object detection with YOLOv8. arXiv preprint arXiv:2305.09972 (2023)"},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2007.901238"},{"key":"e_1_2_11_9_1","doi-asserted-by":"crossref","unstructured":"Mairal J. Bach F. Ponce J. Sapiro G. Zisserman A.:Non\u2010local sparse models for image restoration. In:2009 IEEE 12th International Conference on Computer Vision pp.2272\u20132279. Kyoto Japen (2009).https:\/\/doi.org\/10.1109\/ICCV.2009.5459452","DOI":"10.1109\/ICCV.2009.5459452"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2012.2235847"},{"key":"e_1_2_11_11_1","doi-asserted-by":"crossref","unstructured":"Gu S. Zhang L. Zuo W. Feng X.:Weighted nuclear norm minimization with application to image denoising. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.2862\u20132869. Columbus OH USA (2014)","DOI":"10.1109\/CVPR.2014.366"},{"key":"e_1_2_11_12_1","doi-asserted-by":"crossref","unstructured":"Liang J. Cao J. Sun G. Zhang K. Van Gool L. Timofte R.:Swinir: Image restoration using swin transformer. In:Proceedings of the IEEE\/CVF International Conference on Computer Vision pp.1833\u20131844. Montreal BC Canada (2021)","DOI":"10.1109\/ICCVW54120.2021.00210"},{"key":"e_1_2_11_13_1","doi-asserted-by":"crossref","unstructured":"Zamir S.W. Arora A. Khan S. Hayat M. Khan F.S. Yang M.H.:Restormer: Efficient transformer for high\u2010resolution image restoration. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.5728\u20135739. New Orleans LA USA (2022)","DOI":"10.1109\/CVPR52688.2022.00564"},{"key":"e_1_2_11_14_1","unstructured":"Lehtinen J. Munkberg J. Hasselgren J. Laine S. Karras T. Aittala M. Aila T.:Noise2Noise: Learning image restoration without clean data. arXiv preprint arXiv:1803.04189 (2018)"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3088914"},{"key":"e_1_2_11_16_1","doi-asserted-by":"crossref","unstructured":"Zamir S.W. Arora A. Khan S. Hayat M. Khan F.S. Yang M.H. Shao L.:Cycleisp: Real image restoration via improved data synthesis. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.2696\u20132705. Seattlele WA USA (2020)","DOI":"10.1109\/CVPR42600.2020.00277"},{"key":"e_1_2_11_17_1","doi-asserted-by":"crossref","unstructured":"Wei K. Fu Y. Yang J. Huang H.:A physics\u2010based noise formation model for extreme low\u2010light raw denoising. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.2758\u20132767. Seattlele WA USA (2020)","DOI":"10.1109\/CVPR42600.2020.00283"},{"key":"e_1_2_11_18_1","doi-asserted-by":"crossref","unstructured":"Quan Y. Chen M. Pang T. Ji H.:Self2self with dropout: Learning self\u2010supervised denoising from single image. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.1890\u20131898. Seattlele WA USA (2020)","DOI":"10.1109\/CVPR42600.2020.00196"},{"key":"e_1_2_11_19_1","doi-asserted-by":"crossref","unstructured":"Chen J. Chen J. Chao H. Yang M.:Image blind denoising with generative adversarial network based noise modeling. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.3155\u20133164. Salt Lack City Utah USA (2018)","DOI":"10.1109\/CVPR.2018.00333"},{"key":"e_1_2_11_20_1","doi-asserted-by":"crossref","unstructured":"Brooks T. Mildenhall B. Xue T. Chen J. Sharlet D. Barron J.T.:Unprocessing images for learned raw denoising. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.11036\u201311045. Long Beach CA USA (2019)","DOI":"10.1109\/CVPR.2019.01129"},{"key":"e_1_2_11_21_1","unstructured":"Wang Y. Yu J. Zhang J.:Zero\u2010shot image restoration using denoising diffusion null\u2010space model. arXiv preprint arXiv:2212.00490 (2022)"},{"key":"e_1_2_11_22_1","first-page":"23593","article-title":"Denoising diffusion restoration models","volume":"35","author":"Kawar B.","year":"2022","journal-title":"Adv. Neural Inf. Process Syst."},{"key":"e_1_2_11_23_1","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho J.","year":"2020","journal-title":"Adv. Neural Inf. Process Syst."},{"key":"e_1_2_11_24_1","unstructured":"Zhang Y. Li D. Shi X. He D. Song K. Wang X. Li H.:Kbnet: Kernel basis network for image restoration. arXiv preprint arXiv:2303.02881 (2023)"},{"key":"e_1_2_11_25_1","doi-asserted-by":"crossref","unstructured":"Guo S. Yan Z. Zhang K. Zuo W. Zhang L.:Toward convolutional blind denoising of real photographs. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.1712\u20131722. Long Beach CA USA (2019)","DOI":"10.1109\/CVPR.2019.00181"},{"key":"e_1_2_11_26_1","doi-asserted-by":"crossref","unstructured":"Krull A. Buchholz T.O. Jug F.:Noise2void\u2010learning denoising from single noisy images. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.2129\u20132137. Long Beach CA USA (2019)","DOI":"10.1109\/CVPR.2019.00223"},{"key":"e_1_2_11_27_1","doi-asserted-by":"crossref","unstructured":"Ohayon G. Adrai T. Vaksman G. Elad M. Milanfar P.:High perceptual quality image denoising with a posterior sampling cgan. In:Proceedings of the IEEE\/CVF International Conference on Computer Vision pp.1805\u20131813. Montreal BC Canada (2021)","DOI":"10.1109\/ICCVW54120.2021.00207"},{"key":"e_1_2_11_28_1","unstructured":"Oord A.V.D. Dieleman S. Zen H. Simonyan K. Vinyals O. Graves A. Kalchbrenner N. Senior A. Kavukcuoglu K.:Wavenet: A generative model for raw audio.https:\/\/arxiv.org\/abs\/1609.03499(2016)"},{"key":"e_1_2_11_29_1","unstructured":"Prakash M. Krull A. Jug F.:Fully unsupervised diversity denoising with convolutional variational autoencoders.https:\/\/arxiv.org\/abs\/2006.06072(2020)"},{"key":"e_1_2_11_30_1","unstructured":"Lugmayr A. Danelljan M. Timofte R.:NTIRE 2021 learning the super\u2010resolution space challenge. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.596\u2013612. Nashville TN USA (2021)"},{"key":"e_1_2_11_31_1","unstructured":"Sohl\u2010Dickstein J. Weiss E. Maheswaranathan N. Ganguli S.:Deep unsupervised learning using nonequilibrium thermodynamics. In:International Conference on Machine Learning pp.2256\u20132265. Lille France (2015)"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3204461"},{"key":"e_1_2_11_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2023.3238179"},{"key":"e_1_2_11_34_1","doi-asserted-by":"crossref","unstructured":"Zhang K. Li Y. Liang J. Cao J. Zhang Y. Tang H. Van Gool L.:Practical blind denoising via swin\u2010conv\u2010unet and data synthesis. arXiv preprint arXiv:2203.13278 (2022)","DOI":"10.1007\/s11633-023-1466-0"},{"key":"e_1_2_11_35_1","doi-asserted-by":"crossref","unstructured":"Burger H.C. Schuler C.J. Harmeling S.:Image denoising: Can plain neural networks compete with BM3D?In:2012 IEEE Conference on Computer Vision and Pattern Recognition pp.2392\u20132399. Provindence RI (2012).https:\/\/doi.org\/10.1109\/CVPR.2012.6247952","DOI":"10.1109\/CVPR.2012.6247952"},{"key":"e_1_2_11_36_1","doi-asserted-by":"crossref","unstructured":"Tu Z. Talebi H. Zhang H. Yang F. Milanfar P. Bovik A. Li Y.:Maxim: Multi\u2010axis mlp for image processing. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.5769\u20135780. New Orleans LA USA (2022)","DOI":"10.1109\/CVPR52688.2022.00568"},{"key":"e_1_2_11_37_1","doi-asserted-by":"crossref","unstructured":"Valanarasu J.M.J. Patel V.M.:Unext: Mlp\u2010based rapid medical image segmentation network. In:International Conference on Medical Image Computing and Computer\u2010Assisted Intervention pp.23\u201333. Singapore (2022)","DOI":"10.1007\/978-3-031-16443-9_3"},{"key":"e_1_2_11_38_1","first-page":"9204","article-title":"Pay attention to MLPS","volume":"34","author":"Liu H.","year":"2021","journal-title":"Adv. Neural Inf. Process Syst."},{"key":"e_1_2_11_39_1","doi-asserted-by":"crossref","unstructured":"Zhang X. Zhou X. Lin M. Sun J.:Shufflenet: An extremely efficient convolutional neural network for mobile devices. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.6848\u20136856. Salt Lack City Utah USA (2018)","DOI":"10.1109\/CVPR.2018.00716"},{"key":"e_1_2_11_40_1","unstructured":"Loshchilov I. Hutter F.:Decoupled weight decay regularization.https:\/\/arxiv.org\/abs\/1711.05101(2017)"},{"key":"e_1_2_11_41_1","unstructured":"Loshchilov I. Hutter F.:SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:https:\/\/arxiv.org\/abs\/1608.03983(2016)"},{"key":"e_1_2_11_42_1","first-page":"8780","article-title":"Diffusion models beat GANS on image synthesis","volume":"34","author":"Dhariwal P.","year":"2021","journal-title":"Adv. Neural Inf. Process Syst."},{"key":"e_1_2_11_43_1","unstructured":"Song J. Meng C. Ermon S.:Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)"},{"key":"e_1_2_11_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_11_45_1","doi-asserted-by":"crossref","unstructured":"Zhang R. Isola P. Efros A.A. Shechtman E. Wang O.:The unreasonable effectiveness of deep features as a perceptual metric. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.586\u2013595. Salt Lack City Utah USA (2018)","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_11_46_1","doi-asserted-by":"crossref","unstructured":"Menon S. Damian A. Hu S. Ravi N. Rudin C.:Pulse: Self\u2010supervised photo upsampling via latent space exploration of generative models. In:Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp.2437\u20132445. Seattlele WA USA (2020)","DOI":"10.1109\/CVPR42600.2020.00251"},{"key":"e_1_2_11_47_1","doi-asserted-by":"crossref","unstructured":"Agustsson E. Timofte R.:Ntire 2017 challenge on single image super\u2010resolution: Dataset and study. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops pp.126\u2013135. Honolulu HI USA (2017)","DOI":"10.1109\/CVPRW.2017.150"},{"key":"e_1_2_11_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.161"},{"key":"e_1_2_11_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2631888"},{"key":"e_1_2_11_50_1","doi-asserted-by":"crossref","unstructured":"Abdelhamed A. Lin S. Brown M.S.:A high\u2010quality denoising dataset for smartphone cameras. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.1692\u20131700. Salt Lack City Utah USA (2018)","DOI":"10.1109\/CVPR.2018.00182"},{"key":"e_1_2_11_51_1","doi-asserted-by":"crossref","unstructured":"Martin D. Fowlkes C. Tal D. Malik J.:A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In:Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. pp.416\u2013423(2001)","DOI":"10.1109\/ICCV.2001.937655"},{"key":"e_1_2_11_52_1","unstructured":"Franzen R.:Kodak lossless true color image suite.http:\/\/r0k.us\/graphics\/kodak\/(1999). Accessed 24 Oct 2021"},{"key":"e_1_2_11_53_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.3600632"},{"key":"e_1_2_11_54_1","doi-asserted-by":"crossref","unstructured":"Nam S. Hwang Y. Matsushita Y. Kim S.J.:A holistic approach to cross\u2010channel image noise modeling and its application to image denoising. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.1683\u20131691. Las Vegas NV USA (2016)","DOI":"10.1109\/CVPR.2016.186"},{"key":"e_1_2_11_55_1","doi-asserted-by":"publisher","DOI":"10.5201\/ipol.2015.125"}],"container-title":["IET Image Processing"],"original-title":[],"language":"en","deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T11:55:38Z","timestamp":1722945338000},"score":1,"resource":{"primary":{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/10.1049\/ipr2.13122"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,13]]},"references-count":54,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.1049\/ipr2.13122"],"URL":"https:\/\/doi.org\/10.1049\/ipr2.13122","archive":["Portico"],"relation":{},"ISSN":["1751-9659","1751-9667"],"issn-type":[{"type":"print","value":"1751-9659"},{"type":"electronic","value":"1751-9667"}],"subject":[],"published":{"date-parts":[[2024,5,13]]},"assertion":[{"value":"2024-02-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}