Abstract
A diabetic foot ulcer(DFU) is a common chronic complication of diabetes because of the dysfunction of islets or receptors of insulin, and it has a high disability and mortality rate. Measuring diabetic foot ulcers is also one of the popular application areas where computer vision combines with deep learning techniques. However, some remaining defects in these studies prevent them from accurately visualizing the wound of different severity. Based on this, we used a multi-classification model to mark the wounds into five grades according to the Wagner diabetic foot grading method. It segmented the different grades in each different level wound using colorfully nested ring shapes to reflect the gradual change of wound grades. We collected 1426 DFU images, of which 967 had nested labels and 459 were single-level labels, with images marked with colored rings to show different degrees of wounds. And then, we constructed a deep learning model of diabetes foot ulcer wounds for semantic segmentation based on Mask Region-based convolutional neural networks (Mask R-CNN), and obtain different levels of diabetes nested segmentation results to reflect the different severity in one wound. Finally, we test and evaluate the performance data of the model. Compared with the state-of- the-art results concerning segmentation and classification and diagnosis of diabetic foot wounds, our model has achieved better performance data (specificity = 99.50%, sensitivity = 70.62%, precision = 84.56%, Mean Average Precision = 85.70%). It shows the effectiveness of our nested segmentation and multi-level classification method. It provides some suggestions and directions for the subsequent evaluation and diagnosis and treatment of diabetic foot ulcers.
Similar content being viewed by others
References
Abdissa D, Adugna T, Gerema U, Dereje D (2020) Prevalence of diabetic foot ulcer and associated factors among adult diabetic patients on follow-up clinic at jimma medical center, southwest ethiopia, 2019: an institutional-based cross-sectional study. J Diabetes Res 2020:1–6. https://doi.org/10.1155/2020/4106383
Al-Rubeaan K, Almashouq MK, Youssef AM, Al- Qumaidi H, Al Derwish M, Ouizi S, Al-Shehri K, Masoodi SN (2017) All-cause mortality among diabetic foot patients and related risk factors in saudi arabia. PLoS ONE 12(11):e0188097. https://doi.org/10.1371/journal.pone.0188097
Alsabek MB, Abdul AR, Aziz (2022) Diabetic foot ulcer, the effect of resource-poor environments on healing time and direct cost: a cohort study during syrian crisis. Int Wound J 19(3):531–537. https://doi.org/10.1111/iwj.13651
Cassidy B, Reeves ND, Pappachan JM, Gillespie D, OShea C, Rajbhandari S, Maiya AG, Frank E, Boulton AJ, Armstrong DG, Najafi B, Wu J, Moi HY (2021) The DFUC 2020 dataset: analysis towards diabetic foot ulcer detection. touchREV Endocrinol 17(1):5–11. https://doi.org/10.17925/EE.2021.17.1.5
Chiao J-Y, Chen K-Y, Liao KY-K, Hsieh P-H, Zhang G, Huang T-C (2019) Detection and classification the breast tumors using Mask R-CNN on sonograms. Medicine 98(19):e15200. https://doi.org/10.1097/MD.0000000000015200
Cui C, Thurnhofer-Hemsi K, Soroushmehr R, Mishra A, Gryak J, Domínguez E, Najarian K, López-Rubio E (2019) Diabetic wound segmentation using convolutional neural networks. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 1002–1005. https://doi.org/10.1109/EMBC.2019.8856665
Dmitriyeva M, Kozhakhmetova Z, Urazova S, Kozhakhmetov S, Turebayev D, Toleubayev M (2022) Inflammatory biomarkers as predictors of infected diabetic foot ulcer. Curr Diabetes Rev 18(6):4–11. https://doi.org/10.2174/1573399817666210928144706
Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE inter- national conference on computer vision, pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587. https://doi.org/10.1109/CVPR.2014.81
Gourishetti K, Keni R, Nayak PG, Jitta SR, Bhaskaran NA, Kumar L, Kumar N, Krishnadas N, Shenoy RR (2020) Sesamol-loaded plga nanosuspension for accelerating wound healing in diabetic foot ulcer in rats. Int J Nanomed 15:9265–9282. https://doi.org/10.2147/IJN.S268941
Goyal M, Yap MH, Reeves ND, Rajbhandari S, Spragg J (2017) Fully convolutional networks for diabetic foot ulcer segmentation. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), pp 618–623. https://doi.org/10.1109/SMC.2017.8122675
Goyal M, Reeves ND, Rajbhandari S, Yap MH (2018) Robust methods for real-time diabetic foot ulcer detection and localization on mobile devices. IEEE J Biomed Health Inf 23(4):1730–1741. https://doi.org/10.1109/JBHI.2018.2868656
Goyal M, Reeves ND, Davison AK, Rajbhandari S, Spragg J, Yap MH (2018) DFUnet: convolutional neural networks for diabetic foot ulcer classification. IEEE Trans Emerg Topics Comput Intell 4(5):728–739. https://doi.org/10.1109/TETCI.2018.2866254
Goyal M, Reeves ND, Rajbhandari S, Ahmad N, Wang C, Yap MH (2020) Recognition of ischaemia and infection in diabetic foot ulcers: dataset and techniques. Comput Biol Med 117:103616. https://doi.org/10.1016/j.compbiomed.2020.103616
Gribbon KT, Bailey DG (2004) A novel approach to real-time bilinear interpolation. In: Proceedings. DELTA 2004. Second IEEE international workshop on electronic design, test and applications, pp 126–131. https://doi.org/10.1109/DELTA.2004.10055
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969. https://doi.org/10.1109/ICCV.2017.322
Joulin A, Cissé M, Grangier D, Jégou H et al (2017) Efficient softmax approximation for GPUs. In: Proceedings of international conference on machine learning, pp 1302–1310. https://doi.org/10.48550/arXiv.1609.04309
Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. Proceedings of international conference for learning representations, pp 1–15. https://doi.org/10.48550/arXiv.1412.6980
Kouitcheu Mabeku LB, Noundjeu Ngamga ML, Leundji H (2020) Helicobacter pylori infection, a risk factor for type 2 diabetes mellitus: a hospital-based cross-sectional study among dyspeptic patients in doualacameroon. Sci Rep 10(1):1–11. https://doi.org/10.1038/s41598-020-69208-3
Lavery LA, Armstrong DG, Harkless LB (1996) Classification of diabetic foot wounds. J Foot Ankle Surg 35(6):528–531. https://doi.org/10.1016/s1067-2516(96)80125-6
LeCun Y, Boser B, Denker J, Henderson D, Howard R, Hubbard W, Jackel L (1989) Handwritten digit recognition with a back-propagation network. Adv Neural Inf Process Syst 2:396–404. https://doi.org/10.5555/2969830.2969879
Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125. https://doi.org/10.1109/CVPR.2017.106
Lu C, Kong Y, Guan Z (2020) A Mask R-CNN model for reidentifying extratropical cyclones based on quasi-supervised thought. Sci Rep 10(1):1–9. https://doi.org/10.1038/s41598-020-71831-z
Malerbi FK, Mendes G, Barboza N, Morales PH, Montargil R, Andrade RE (2022) Diabetic macular edema screened by handheld smartphone-based retinal camera and artificial intelligence. J Med Syst 46(1):1–6. https://doi.org/10.1007/s10916-021-01795-8
Ohura N, Mitsuno R, Sakisaka M, Terabe Y, Morishige Y, Uchiyama A, Okoshi T, Shinji I, Takushima A (2019) Convolutional neural networks for wound detection: the role of artificial intelligence in wound care. J Wound Care 28(10):13–24. https://doi.org/10.12968/jowc.2019.28.Sup10.S13
Rastogi A, Goyal G, Kesavan R, Bal A, Kumar H, Kamath P, Jude EB, Armstrong DG, Bhansali A et al (2020) Long term outcomes after incident diabetic foot ulcer: Multicenter large cohort prospective study (edi-focus investigators) epidemiology of diabetic foot complications study: epidemiology of diabetic foot complications study. Diabetes Res Clin Pract 162:108–113. https://doi.org/10.1016/j.diabres.2020.108113
Rebecca R, Dominic S, Boyoung K, James M, Williams D, Leslie A (2020) The diabetic foot ulcer. Australian J Gen Practitioners 49(5):250–255. https://doi.org/10.31128/AJGP-11-19-5161
Ren S, He K, Girshick R, Sun J, Faster R-CNN (2015) Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99. https://doi.org/10.1109/TPAMI.2016.2577031
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Proceedings of International Conference on Medical image computing and computer-assisted intervention, pp 234–241. https://doi.org/10.1007/978-3-319-24574-428
Rümenapf G, Morbach S, Rother U, Uhl C, Görtz H, Böckler D, Behrendt C-A, Hochlenert D, Engels G, Sigl M et al (2021) Diabetisches fußsyndrom–teil 1: definition, pathophysiologie, diagnostik und klassifikation, Der Chirurg. Z Fur Alle Gebiete Der Operativen Medizen 92(1):81–94. https://doi.org/10.1007/s00104-020-01301-9
Syed MH, Salata K, Hussain MA, Zamzam A, de Mestral C, Wheatcroft M, Harlock J, Awartani D, Aljabri B, Verma A et al (2020) The economic burden of inpatient diabetic foot ulcers in Toronto, Canada. Vascular 28(5):520–529. https://doi.org/10.1177/1708538120923420
Wagner FW Jr (1981) The dysvascular foot: a system for diagnosis and treatment. Foot Ankle 2(2):64–122. https://doi.org/10.1177/107110078100200202
Wang P, Ge R, Xiao X, Cai Y, Wang G, Zhou F (2017) Rectified- linear-unit-based deep learning for biomedical multi-label data. Interdiscip Sci Comput Life Sci 9(3):419–422. https://doi.org/10.1007/s12539-016-0196-1
Wang T, Zhong L, Yuan J, Wang T, Yin S, Sun Y, Li- u X, Liu X, Ling S (2020) Quantitative analysis of functional filtering bleb size using Mask R-CNN. Ann Transl Med 8(11):709–717. https://doi.org/10.21037/atm.2020.03.135
Woldemariam GT, Atnafu NT, Radie YT, Wolde GT, Gebreagziabher TT, Gebrehiwot TG, Teka YH, Hagos MG, Hagezom HM, Yigzaw HB et al (2020) Determinants of diabetic foot ulcer among adult patients with diabetes attending the diabetic clinic in tikur anbessa specialized hospital, addis ababa, ethiopia: Unmatched case–control study. Diabetes Metab Syndr Obes: Targets Ther 13:3739–3747. https://doi.org/10.2147/DMSO.S265988
Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500. https://doi.org/10.1109/CVPR.2017.634
Yap MH, Hachiuma R, Alavi A, Brüngel R, Cassidy B, Goyal M, Zhu H, Rückert J, Olshansky M, Huang X et al (2021) Deep learning in diabetic foot ulcers detection: a comprehensive evaluation. Comput Biol Med 135:104596. https://doi.org/10.1016/j.compbiomed.2021.104596
Yap MH, Cassidy B, Pappachan JM, O’Shea C, Gillespie D, Reeves N (2021) Analysis towards classification of infection and ischaemia of diabetic foot ulcers. In: 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), pp 1–4. https://doi.org/10.1109/BHI50953.2021.9508563
Acknowledgements
We are grateful to all patients involved in all trials and investigators for their previous work that enabled the present study. This study was supported by the Hunan Province Natural Science Foundation (grant number 2022JJ30673), Scientific Research Fund of Hunan Provincial Education Department (grant number 20C0402), Hunan First Normal University (grant number XYS16N03), the Projects of the National Natural Science Foundation of China (grant number 82073018), the Shenzhen Science and Technology Innovation Commission (Natural Science Foundation of Shenzhen, grant number JCYJ20210324113001005), Management Research Fund of Xiangya Hospital of Central South University (grant number 2021GL11).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare no competing interests.
I confirm I have included a data availability statement in my main manuscript file.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cao, C., Qiu, Y., Wang, Z. et al. Nested segmentation and multi-level classification of diabetic foot ulcer based on mask R-CNN. Multimed Tools Appl 82, 18887–18906 (2023). https://doi.org/10.1007/s11042-022-14101-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-14101-6