Automatic and visualized grading of dental caries using deep learning on panoramic radiographs | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

Automatic and visualized grading of dental caries using deep learning on panoramic radiographs

  • Track 2: Medical Applications of Multimedia
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Caries grading plays a significant role for oral health management and treatment planning. Grading caries on panoramic image is a challenging task due to complication and diversity of gray distribution. In this paper, we proposed an automatic and visualized caries grading method for panoramic image using deep learning-based tooth anatomical segmentation and regions intersection judgment to achieve a consistent grading process with dentist. To achieve accurate semantic segmentation, a modified U-Net model by adding ASPP module and boundary loss is applied to segment caries, enamel, dentin, and pulp tissue region. Then a visualized process is conducted to judge the intersection of carious region and decision-making line for grading of shallow, medium, deep caries. Experimental results demonstrate our method achieves promising grading performance. Moreover, we validated that our proposed two-stage caries grading method outperform deep learning classification models. Ablation analysis of anatomical segmentation performance was also investigated, and the compared results show that our proposed modified U-Net model can obtain more accurate region and boundary to improve grading results. Some mis-graded cases were finally detailed analyzed. Our proposed caries grading approach has great potential for clinical aided diagnosis and automatic chart filling on panoramic radiographs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The datasets analysed during the current study are not publicly available due privacy but are available from the corresponding author on reasonable request.

References

  1. Black GV (1917) A Work on operative dentistry v. 2. Medico-dental publishing Company 2, Chicago

    Google Scholar 

  2. Caliva F, Iriondo C, Martinez AM, Majumdar S, Pedoia V (2019) Distance map loss penalty term for semantic segmentation. arXiv preprint arXiv:1908.03679

  3. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

  4. Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587

  5. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  6. Chen X, Williams BM, Vallabhaneni SR, Czanner G, Williams R, Zheng Y (2019) Learning active contour models for medical image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11632–11640

  7. Das S, Kharbanda K, Suchetha M, Raman R, Dhas E (2021) Deep learning architecture based on segmented fundus image features for classification of diabetic retinopathy. Biomed Signal Process Control 68:102600

    Article  Google Scholar 

  8. Fang L, Wang C, Li S, Rabbani H, Chen X, Liu Z (2019) Attention to lesion: lesion-aware convolutional neural network for retinal optical coherence tomography image classification. IEEE Trans Med Imaging 38(8):1959–1970

  9. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3146–3154

  10. Gao X, Lin S, Wong TY (2015) Automatic feature learning to grade nuclear cataracts based on deep learning. IEEE Trans Biomed Eng 62(11):2693–2701

    Article  Google Scholar 

  11. Goldberg M (2016) Understanding dental caries. Springer, Switzerland

    Book  Google Scholar 

  12. Goldberg M (2020) Enamel and dentin carious lesions. JSM Dent 8(1):11–20

    Google Scholar 

  13. Haghanifar A, Majdabadi MM, Ko S-B (2020) Paxnet: dental caries detection in panoramic x-ray using ensemble transfer learning and capsule classifier. arXiv preprint arXiv:2012.13666

  14. Han Y, Li X, Wang B, Wang L (2021) Boundary loss-based 2.5 d fully convolutional neural networks approach for segmentation: a case study of the liver and tumor on computed tomography. Algorithms 14(5):144

    Article  Google Scholar 

  15. Hwang J-J, Jung Y-H, Cho B-H, Heo M-S (2019) An overview of deep learning in the field of dentistry. Imaging Sci Dent 49(1):1–7

    Article  Google Scholar 

  16. Ismail AI, Sohn W, Tellez M, Amaya A, Sen A, Hasson H, Pitts NB (2007) The international caries detection and assessment system (icdas): an integrated system for measuring dental caries. Commun Dent Oral Epidemiol 35(3):170–178

    Article  Google Scholar 

  17. Ismail AI, Pitts NB, Tellez M (2015) The international caries classification and management system (iccms™) an example of a caries management pathway. BMC Oral Health 15(1):1–13

    Google Scholar 

  18. Jader G, Fontineli J, Ruiz M, Abdalla K, Pithon M, Oliveira L (2018) Deep instance segmentation of teeth in panoramic x-ray images. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, pp 400–407

  19. Jadon S (2020) A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB). IEEE, pp 1–7

  20. Jurdi RE, Petitjean C, Honeine P, Cheplygina V, Abdallah F (2021) A surprisingly effective perimeter-based loss for medical image segmentation. In: Medical imaging with deep learning. PMLR, pp 158–167

  21. Karimi D, Salcudean SE (2019) Reducing the hausdorff distance in medical image segmentation with convolutional neural networks. IEEE Trans Med Imaging 39(2):499–513

    Article  Google Scholar 

  22. Kassebaum N, Bernabé E, Dahiya M, Bhandari B, Murray C, Marcenes W (2015) Global burden of untreated caries: a systematic review and metaregression. J Dent Res 94(5):650–658

    Article  Google Scholar 

  23. Kervadec H, Bouchtiba J, Desrosiers C, Granger E, Dolz J, Ayed IB (2019) Boundary loss for highly unbalanced segmentation. International conference on medical imaging with deep learning. PMLR, pp 285–296

  24. Koch TL, Perslev M, Igel C, Brandt SS (2019) Accurate segmentation of dental panoramic radiographs with u-nets. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, pp 15–19

  25. Lee J-H, Kim D-H, Jeong S-N, Choi S-H (2018) Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm. J Dent 77:106–111

    Article  Google Scholar 

  26. Leo LM, Reddy TK (2021) Learning compact and discriminative hybrid neural network for dental caries classification. Microprocess Microsyst 82:103836

    Article  Google Scholar 

  27. Li Y, Huang M, Zhang Y, Chen J, Xu H, Wang G, Feng W (2020) Automated gleason grading and gleason pattern region segmentation based on deep learning for pathological images of prostate cancer. IEEE Access 8:117714–117725

    Article  Google Scholar 

  28. Lin D, Ji Y, Lischinski D, Cohen-Or D, Huang H (2018) Multi-scale context intertwining for semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 603–619

  29. Martinez-Murcia FJ, Ortiz A, Ramírez J, Górriz JM, Cruz R (2021) Deep residual transfer learning for automatic diagnosis and grading of diabetic retinopathy. Neurocomputing 452:424–434

    Article  Google Scholar 

  30. Mohan G, Subashini MM (2018) MRI based medical image analysis: Survey on brain tumor grade classification. Biomed Signal Process Control 39:139–161

    Article  Google Scholar 

  31. Mount G, Tyas M, Duke E, Lasfargues J, Kaleka R, Hume W (2006) A proposal for a new classification of lesions of exposed tooth surfaces. Int Dent J 56(2):82–91

    Article  Google Scholar 

  32. Nagpal K, Foote D, Liu Y, Chen P-HC, Wulczyn E, Tan F, Olson N, Smith JL, Mohtashamian A, Wren JH et al (2019) Development and validation of a deep learning algorithm for improving gleason scoring of prostate cancer. NPJ Digit Med 2(1):1–10

    Google Scholar 

  33. Naser MA, Deen MJ (2020) Brain tumor segmentation and grading of lower-grade glioma using deep learning in mri images. Comput Biol Med 121:103758

    Article  Google Scholar 

  34. Qiu M, Zhang C, Song Z (2022) Dynamic boundary-insensitive loss for magnetic resonance medical image segmentation. Med Phys 49(3):1739–1753

    Article  Google Scholar 

  35. Qureshi I, Ma J, Abbas Q (2021) Diabetic retinopathy detection and stage classification in eye fundus images using active deep learning. Multimed Tools Appl 80(8):11691–11721

    Article  Google Scholar 

  36. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, pp 234–241

  37. Sajjad M, Khan S, Muhammad K, Wu W, Ullah A, Baik SW (2019) Multi-grade brain tumor classification using deep CNN with extensive data augmentation. J Comput Sci 30:174–182

    Article  Google Scholar 

  38. Silva G, Oliveira L, Pithon M (2018) Automatic segmenting teeth in x-ray images: trends, a novel data set, benchmarking and future perspectives. Expert Syst Appl 107:15–31

  39. Singh P, Sehgal P (2021) GV black dental caries classification and preparation technique using optimal cnn-lstm classifier. Multimed Tools Appl 80(4):5255–5272

    Article  Google Scholar 

  40. Sinha A, Dolz J (2020) Multi-scale self-guided attention for medical image segmentation. IEEE J Biomed Health Inf 25(1):121–130

    Article  Google Scholar 

  41. Stidham RW, Liu W, Bishu S, Rice MD, Higgins PD, Zhu J, Nallamothu BK, Waljee AK (2019) Performance of a deep learning model vs human reviewers in grading endoscopic disease severity of patients with ulcerative colitis. JAMA Netw open 2(5):e193963–e193963

    Article  Google Scholar 

  42. Sultan HH, Salem NM, Al-Atabany W (2019) Multi-classification of brain tumor images using deep neural network. IEEE Access 7:69215–69225

    Article  Google Scholar 

  43. Tao A, Sapra K, Catanzaro B (2020) Hierarchical multiscale attention for semantic segmentation. arXiv preprint arXiv:2005.10821

  44. Tran S-T, Cheng C-H, Nguyen T-T, Le M-H, Liu D-G (2021) Tmd-unet: triple-unet with multi-scale input features and dense skip connection for medical image segmentation. Healthcare 9(1):54. Multidisciplinary Digital Publishing Institute

  45. Tuzoff DV, Tuzova LN, Bornstein MM, Krasnov AS, Kharchenko MA, Nikolenko SI, Sveshnikov MM, Bednenko GB (2019) Tooth detection and numbering in panoramic radiographs using convolutional neural networks. Dentomaxillofacial Radiol 48(4):20180051

    Article  Google Scholar 

  46. Vinayahalingam S, Kempers S, Limon L, Deibel D, Maal T, Hanisch M, Bergé S, Xi T (2021) Classification of caries in third molars on panoramic radiographs using deep learning. Sci Rep 11(1):1–7

    Article  Google Scholar 

  47. Wang Y, Yu M, Hu B, Jin X, Li Y, Zhang X, Zhang Y, Gong D, Wu C, Zhang B et al (2021) Deep learning-based detection and stage grading for optimising diagnosis of diabetic retinopathy. Diabetes Metab Res Rev 37(4):e3445

    Article  Google Scholar 

  48. Wang X, Li Z, Huang Y, Jiao Y (2022) Multimodal medical image segmentation using multi-scale context-aware network. Neurocomputing 486:135–146

    Article  Google Scholar 

  49. Whaites E, Drage N (2013) Essentials of dental radiography and radiology. Elsevier Health Sciences, London

    Google Scholar 

  50. Xu X, Zhang L, Li J, Guan Y, Zhang L (2019) A hybrid global-local representation cnn model for automatic cataract grading. IEEE J Biomed Health Inf 24(2):556–567

    Article  Google Scholar 

  51. Yang D, Wang J, Lu G (2002) An algorithm for automatically generating centerlines between curves(in Chinese). Bull Surveying Mapp 3:58–60

  52. Yang Q, Ku T, Hu K (2021) Efficient attention pyramid network for semantic segmentation. IEEE Access 9:18867–18875

    Article  Google Scholar 

  53. Yang Q, Xu Z, Liao C, Cai J, Huang Y, Chen H, Tao X, Huang Z, Chen J, Dong J et al (2020) Epithelium segmentation and automated gleason grading of prostate cancer via deep learning in label-free multiphoton microscopic images. J Biophotonics 13(2):e201900203

    Article  Google Scholar 

  54. Zeller G, Young DA, Novy B (2019) The american dental association caries classification system (ada ccs). In: Detection and assessment of dental caries. Springer, pp 57–67

  55. Zhang J, Jin Y, Xu J, Xu X, Zhang Y (2018) Mdu-net: multiscale densely connected u-net for biomedical image segmentation. arXiv preprint arXiv:1812.00352

  56. Zhang X, Liang Y, Li W, Liu C, Gu D, Sun W, Miao L (2022) Development and evaluation of deep learning for screening dental caries from oral photographs. Oral Dis 28(1):173–181

    Article  Google Scholar 

  57. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  58. Zhao S, Wang Y, Yang Z, Cai D (2019) Region mutual information loss for semantic segmentation. Adv Neural Inf Process Syst 32

  59. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J (2018) Unet++: a nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, pp 3–11

  60. Zhu H, Cao Z, Lian L et al (2022) CariesNet: a deep learning approach for segmentation of multi-stage caries lesion from oral panoramic X-ray image. Neural Comput Appl:1–9

Download references

Acknowledgements

This work was supported by the Fundamental Research Funds for the Zhejiang Provincial Universities (2021XZZX033), Natural Science Foundation of Zhejiang Province (LZY21F030002), Department of Science and Technology of Zhejiang Province (Y202045833).

Author information

Authors and Affiliations

  1. School of Automation, Hangzhou Dianzi University, Hangzhou, China, 310018

    Qingguang Chen, Junchao Huang, Kaihua Wei & Xiaomin Lai

  2. Affiliated Hospital of Stomatology of Zhejiang University, Hangzhou, China, 310018

    Haihua Zhu & Luya Lian

Authors
  1. Qingguang Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Junchao Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Haihua Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Luya Lian
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Kaihua Wei
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Xiaomin Lai
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qingguang Chen.

Ethics declarations

The researcher claims no conflicts of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Q., Huang, J., Zhu, H. et al. Automatic and visualized grading of dental caries using deep learning on panoramic radiographs. Multimed Tools Appl 82, 23709–23734 (2023). https://doi.org/10.1007/s11042-022-14089-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14089-z

Keywords