An attention enriched encoder–decoder architecture with CLSTM and RES unit for segmenting exudate in retinal images | Signal, Image and Video Processing Skip to main content

Advertisement

Springer Nature Link
Log in

An attention enriched encoder–decoder architecture with CLSTM and RES unit for segmenting exudate in retinal images

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Diabetic retinopathy, an eye complication that causes retinal damage, can impair the vision and even result in blindness, if not treated on time. Regular eye screening is essential for patients with diabetics because diabetic retinopathy advances significantly without symptoms. Exudates are a primary symptom of diabetic retinopathy, and their automatic recognition can help in early diagnosis. The convolution operation which concentrates mostly on extracting the local features provides less emphasis on global information resulting the long-range dependencies to be addressed while traversing through multiple layers. The proposed segmentation model utilizes both the channel and spatial attention mechanisms to effectively establish the long-range dependencies at various levels of feature extraction. The proposed methodology also utilizes the convolutional long- and short-term memory algorithm during the propagation from input-to-state and from the state-to-state to take into account the spatiotemporal dependencies and the residual extended skip block for widening the network's receptive zone. Implementing the potentials of neural networks, this study excels at identifying complex patterns and minute features in retinal images. The effectiveness of the proposed method has been verified by conducting experiments on various retinal image datasets, such as IDRiD, MESSIDOR, DIARETDB0, and DIARETDB1, which clearly indicates the superiority of this method over other existing methods across a wide range of evaluation metrics, namely specificity, F1-score, accuracy, sensitivity, and intersection-over-union. Additionally, the model's ability to achieve an overall accuracy of 97.7% makes it a viable application that can provide clinicians important insights into the diagnosis and treatment of diabetic retinopathy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data and materials

The data generated during this study are available from the corresponding author on reasonable request.

References

  1. Ali, S., Sidibé, D., Adal, K.M., Giancardo, L., Chaum, E., Karnowski, T.P., Mériaudeau, F.: Statistical atlas based exudate segmentation. Comput. Med. Imaging Graph. 37, 358–368 (2013)

    Article  Google Scholar 

  2. Harangi, B., Hajdu, A.: Detection of exudates in fundus images using a Markovian segmentation model. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 130–133. IEEE (2014)

  3. Pereira, C., Gonçalves, L., Ferreira, M.: Exudate segmentation in fundus images using an ant colony optimization approach. Inf. Sci. (N.Y.) 296, 14–24 (2015)

    Article  MathSciNet  Google Scholar 

  4. Sánchez, C.I., García, M., Mayo, A., López, M.I., Hornero, R.: Retinal image analysis based on mixture models to detect hard exudates. Med. Image Anal. 13, 650–658 (2009)

    Article  Google Scholar 

  5. Sopharak, A., Uyyanonvara, B., Barman, S., Williamson, T.H.: Automatic detection of diabetic retinopathy exudates from non-dilated retinal images using mathematical morphology methods. Comput. Med. Imaging Graph. 32, 720–727 (2008)

    Article  Google Scholar 

  6. Chen, Y., Xia, R., Zou, K., Yang, K.: FFTI: Image inpainting algorithm via features fusion and two-steps inpainting. J. Vis. Commun. Image Represent. 91, 103776 (2023)

    Article  Google Scholar 

  7. Chen, Y., Xia, R., Zou, K., Yang, K.: RNON: Image inpainting via repair network and optimization network. Int. J. Mach. Learn. Cybern. 14, 1–17 (2023)

    Article  Google Scholar 

  8. Chen, Y., Xia, R., Yang, K., Zou, K.: DARGS: Image inpainting algorithm via deep attention residuals group and semantics. J. King Saud Univ. Inf. Sci. 35, 101567 (2023)

    Google Scholar 

  9. Chen, Y., Xia, R., Yang, K., Zou, K.: DGCA: high resolution image inpainting via DR-GAN and contextual attention. Multimed. Tools Appl. 82, 1–21 (2023)

    Article  Google Scholar 

  10. Giancardo, L., Meriaudeau, F., Karnowski, T.P., Li, Y., Garg, S., Tobin, K.W., Jr., Chaum, E.: Exudate-based diabetic macular edema detection in fundus images using publicly available datasets. Med. Image Anal. 16, 216–226 (2012)

    Article  Google Scholar 

  11. García, M., Sánchez, C.I., López, M.I., Abásolo, D., Hornero, R.: Neural network based detection of hard exudates in retinal images. Comput. Methods Programs Biomed. 93, 9–19 (2009)

    Article  Google Scholar 

  12. Zhang, X., Thibault, G., Decencière, E., Marcotegui, B., Laÿ, B., Danno, R., Cazuguel, G., Quellec, G., Lamard, M., Massin, P.: Exudate detection in color retinal images for mass screening of diabetic retinopathy. Med. Image Anal. 18, 1026–1043 (2014)

    Article  Google Scholar 

  13. Fraz, M.M., Jahangir, W., Zahid, S., Hamayun, M.M., Barman, S.A.: Multiscale segmentation of exudates in retinal images using contextual cues and ensemble classification. Biomed. Signal Process. Control 35, 50–62 (2017)

    Article  Google Scholar 

  14. Chen, Y., Xia, R., Yang, K., Zou, K.: MFFN: image super-resolution via multi-level features fusion network. Vis. Comput. 40, 1–16 (2023)

    Google Scholar 

  15. Bilal, A., Zhu, L., Deng, A., Lu, H., Wu, N.: AI-based automatic detection and classification of diabetic retinopathy using U-Net and deep learning. Symmetry (Basel) 14, 1427 (2022)

    Article  Google Scholar 

  16. Bilal, A., Sun, G., Mazhar, S., Imran, A., Latif, J.: A transfer learning and U-Net-based automatic detection of diabetic retinopathy from fundus images. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 10, 663–674 (2022)

    Article  Google Scholar 

  17. Bilal, A., Sun, G., Mazhar, S.: Diabetic retinopathy detection using weighted filters and classification using CNN. In: 2021 International Conference on Intelligent Technologies (CONIT), pp. 1–6. IEEE (2021)

  18. Bilal, A., Sun, G., Mazhar, S.: Survey on recent developments in automatic detection of diabetic retinopathy. J. Fr. Ophtalmol. 44, 420–440 (2021)

    Article  Google Scholar 

  19. Bilal, A., Sun, G., Li, Y., Mazhar, S., Khan, A.Q.: Diabetic retinopathy detection and classification using mixed models for a disease grading database. IEEE Access 9, 23544–23553 (2021)

    Article  Google Scholar 

  20. Bilal, A., Sun, G., Mazhar, S., Imran, A.: Improved grey wolf optimization-based feature selection and classification using CNN for diabetic retinopathy detection. In: Evolutionary Computing and Mobile Sustainable Networks: Proceedings of ICECMSN 2021, pp. 1–14. Springer (2022)

  21. Saha, S.K., Fernando, B., Xiao, D., Tay-Kearney, M.-L., Kanagasingam, Y.: Deep learning for automatic detection and classification of microaneurysms, hard and soft exudates, and hemorrhages for diabetic retinopathy diagnosis. Investig. Ophthalmol. Vis. Sci. 57, 5962 (2016)

    Google Scholar 

  22. Maiti, S., Maji, D., Dhara, A.K., Sarkar, G.: Automatic detection and segmentation of optic disc using a modified convolution network. Biomed. Signal Process. Control 76, 103633 (2022)

    Article  Google Scholar 

  23. Tan, J.H., Fujita, H., Sivaprasad, S., Bhandary, S.V., Rao, A.K., Chua, K.C., Acharya, U.R.: Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf. Sci. (N.Y.) 420, 66–76 (2017)

    Article  Google Scholar 

  24. Prentašić, P., Lončarić, S.: Detection of exudates in fundus photographs using deep neural networks and anatomical landmark detection fusion. Comput. Methods Programs Biomed. 137, 281–292 (2016)

    Article  Google Scholar 

  25. Yu, S., Xiao, D., Kanagasingam, Y.: Exudate detection for diabetic retinopathy with convolutional neural networks. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1744–1747. IEEE (2017)

  26. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  27. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer (2015)

  28. Alom, M.Z., Yakopcic, C., Hasan, M., Taha, T.M., Asari, V.K.: Recurrent residual U-Net for medical image segmentation. J. Med. Imaging 6, 14006 (2019)

    Article  Google Scholar 

  29. Chen, W., Zhang, Y., He, J., Qiao, Y., Chen, Y., Shi, H., Wu, E.X., Tang, X.: Prostate segmentation using 2D bridged U-net. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2019)

  30. Nie, D., Gao, Y., Wang, L., Shen, D.: ASDNet: attention based semi-supervised deep networks for medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 370–378. Springer (2018)

  31. Roy, A.G., Navab, N., Wachinger, C.: Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 421–429. Springer (2018)

  32. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. https://arxiv.org/abs/1409.0473 (2014)

  33. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  34. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

  35. Zhao, J., Mao, X., Chen, L.: Speech emotion recognition using deep 1D & 2D CNN LSTM networks. Biomed. Signal Process. Control 47, 312–323 (2019)

    Article  Google Scholar 

  36. Azad, R., Asadi-Aghbolaghi, M., Fathy, M., Escalera, S.: Bi-directional ConvLSTM U-Net with densley connected convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)

  37. Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Deep Learning and Data Labeling for Medical Applications, pp. 179–187. Springer (2016)

  38. Fang, W., Han, X.: Spatial and channel attention modulated network for medical image segmentation. In: Proceedings of the Asian Conference on Computer Vision (2020)

  39. Sherstinsky, A.: Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Phys. D Nonlinear Phenom. 404, 132306 (2020)

    Article  MathSciNet  Google Scholar 

  40. Liu, Y., Zheng, H., Feng, X., Chen, Z.: Short-term traffic flow prediction with Conv-LSTM. In: 2017 9th International Conference on Wireless Communications and Signal Processing (WCSP), pp. 1–6. IEEE (2017)

  41. Rehman, M.U., Cho, S., Kim, J.H., Chong, K.T.: Bu-net: brain tumor segmentation using modified u-net architecture. Electronics 9, 2203 (2020)

    Article  Google Scholar 

  42. Porwal, P., Pachade, S., Kamble, R., Kokare, M., Deshmukh, G., Sahasrabuddhe, V., Meriaudeau, F.: Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research. Data 3, 25 (2018)

    Article  Google Scholar 

  43. Decencière, E., Zhang, X., Cazuguel, G., Lay, B., Cochener, B., Trone, C., Gain, P., Ordonez, R., Massin, P., Erginay, A.: Feedback on a publicly distributed image database: the Messidor database. Image Anal. Stereol. 33, 231–234 (2014)

    Article  Google Scholar 

  44. Kauppi, T., Kalesnykiene, V., Kamarainen, J.-K., Lensu, L., Sorri, I., Uusitalo, H., Kälviäinen, H., Pietilä, J.: DIARETDB0: evaluation database and methodology for diabetic retinopathy algorithms. In: Machine Vision and Pattern Recognition Research Group, Lappeenranta University of Technology, Finland, vol. 73, pp. 1–17 (2006)

  45. Kälviäinen, R., Uusitalo, H.: DIARETDB1 diabetic retinopathy database and evaluation protocol. In: Medical Image Understanding and Analysis, p. 61. Citeseer (2007)

  46. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)

    Article  Google Scholar 

  47. Kolhar, S., Jagtap, J.: Convolutional neural network based encoder–decoder architectures for semantic segmentation of plants. Ecol. Inform. 64, 101373 (2021)

    Article  Google Scholar 

  48. Zabihollahy, F., Lochbihler, A., Ukwatta, E.: Deep learning based approach for fully automated detection and segmentation of hard exudate from retinal images. In: Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging, pp. 17–22. SPIE (2019)

  49. Guo, S., Wang, K., Kang, H., Liu, T., Gao, Y., Li, T.: Bin loss for hard exudates segmentation in fundus images. Neurocomputing 392, 314–324 (2020)

    Article  Google Scholar 

  50. Kaur, J., Mittal, D.: A generalized method for the segmentation of exudates from pathological retinal fundus images. Biocybern. Biomed. Eng. 38, 27–53 (2018)

    Article  Google Scholar 

  51. Agurto, C., Murray, V., Yu, H., Wigdahl, J., Pattichis, M., Nemeth, S., Barriga, E.S., Soliz, P.: A multiscale optimization approach to detect exudates in the macula. IEEE J. Biomed. Heal. Inform. 18, 1328–1336 (2014)

    Article  Google Scholar 

  52. Lokuarachchi, D., Gunarathna, K., Muthumal, L., Gamage, T.: Automated detection of exudates in retinal images. In: 2019 IEEE 15th International Colloquium on Signal Processing & Its Applications (CSPA), pp. 43–47. IEEE (2019)

  53. Akram, M.U., Tariq, A., Anjum, M.A., Javed, M.Y.: Automated detection of exudates in colored retinal images for diagnosis of diabetic retinopathy. Appl. Opt. 51, 4858–4866 (2012)

    Article  Google Scholar 

  54. Khojasteh, P., Júnior, L.A.P., Carvalho, T., Rezende, E., Aliahmad, B., Papa, J.P., Kumar, D.K.: Exudate detection in fundus images using deeply-learnable features. Comput. Biol. Med. 104, 62–69 (2019)

    Article  Google Scholar 

  55. Liu, Q., Zou, B., Chen, J., Ke, W., Yue, K., Chen, Z., Zhao, G.: A location-to-segmentation strategy for automatic exudate segmentation in colour retinal fundus images. Comput. Med. Imaging Graph. 55, 78–86 (2017)

    Article  Google Scholar 

  56. Yazid, H., Arof, H., Isa, H.M.: Exudates segmentation using inverse surface adaptive thresholding. Measurement 45, 1599–1608 (2012)

    Article  Google Scholar 

Download references

Funding

No funds, grants, or other support was received.

Author information

Authors and Affiliations

  1. Institute of Engineering and Management - Newtown, University of Engineering and Management, Kolkata, 700160, India

    Souvik Maiti

  2. Department of Electrical Engineering, Haldia Institute of Technology, Haldia, West Bengal, 721657, India

    Debasis Maji

  3. Department of Electrical Engineering, National Institute of Technology, Durgapur, West Bengal, 713209, India

    Ashis Kumar Dhara

  4. Department of Electrical Engineering, Jadavpur University, Kolkata, 700032, India

    Gautam Sarkar

Authors
  1. Souvik Maiti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debasis Maji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashis Kumar Dhara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gautam Sarkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have contributed equally.

Corresponding author

Correspondence to Souvik Maiti.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maiti, S., Maji, D., Dhara, A.K. et al. An attention enriched encoder–decoder architecture with CLSTM and RES unit for segmenting exudate in retinal images. SIViP 18, 3329–3339 (2024). https://doi.org/10.1007/s11760-024-02996-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-02996-7

Keywords

Search

Navigation