SSANet: spatial stain attention network for pathological images classification | Multimedia Tools and Applications Skip to main content

Advertisement

Springer Nature Link
Log in

SSANet: spatial stain attention network for pathological images classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Histopathological images classification plays a significant role in cancer diagnosis, but current deep learning methods fail to account for the unique characteristics of histopathological images. To address this limitation, we present SSANet, a Spatial and Stain Attention Network focusing on staining information in the cell nucleus and cytoplasm of histopathological images. Our approach first separates the stain channels and generates a stain attention map using a specialized stain attention module, which then activates staining information in the feature map. The experiments on two computational pathology problems, CRC-HE and BreakHis datasets, demonstrate that our method outperforms state-of-the-art methods with test F1 values up to 97.63% and 94.91% for cancer subtypes classification. Our contributions include the novel SSANet architecture, a stain attention module that enhances the focus on crucial cytosolic and cytoplasmic information, and improved classification results for histopathological images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

All data generated or analyzed during this study are included in these published articles: ‘Kather, J.N., Halama, N., Marx, A.: 100,000 histological images of human colorectal cancer and healthy tissue. Zenodo10 5281 (2018)’ [48], ‘Spanhol, F.A., Oliveira, L.S., Petitjean, C., Heutte, L.: A dataset for breast cancer histopathological image classification. Ieee transactions on biomedical engineering 63(7), 1455-1462 (2015)’ [49] and ‘Litjens, Geert, et al.: 1399 H &E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset. GigaScience 7.6 (2018): giy065. https://doi.org/10.1093/gigascience/giy065.’ [55]

References

  1. Ragab DA, Attallah O, Sharkas M, Ren J, Marshall S (2021) A framework for breast cancer classification using multi-dcnns. Computers in Biology and Medicine 131:104245

    Article  PubMed  Google Scholar 

  2. Kumar A, Singh SK, Saxena S, Lakshmanan K, Sangaiah AK, Chauhan H, Shrivastava S, Singh RK (2020) Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer. Information Sciences 508:405–421

    Article  Google Scholar 

  3. Brook, A., El-Yaniv, R., Isler, E., Kimmel, R., Meir, R., Peleg, D.: Breast cancer diagnosis from biopsy images using generic features and svms. IEEE Transactions on Information Technology in Biomedicine (2006)

  4. Filipczuk P, Fevens T, Krzyżak A, Monczak R (2013) Computer-aided breast cancer diagnosis based on the analysis of cytological images of fine needle biopsies. IEEE transactions on medical imaging 32(12):2169–2178

    Article  PubMed  Google Scholar 

  5. George Y, Zayed H, Roushdy M, Elbagoury B (2013) Remote computer-aided breast cancer detection and diagnosis system based on cytological images. IEEE Syst J 8(3):949–964

    Article  ADS  Google Scholar 

  6. Wan, S., Huang, X., Lee, H.-C., Fujimoto, J.G., Zhou, C.: Spoke-lbp and ring-lbp: New texture features for tissue classification. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 195–199 (2015). IEEE

  7. Belsare, A., Mushrif, M., Pangarkar, M., Meshram, N.: Classification of breast cancer histopathology images using texture feature analysis. In: Tencon 2015-2015 IEEE Region 10 Conference, pp. 1–5 (2015). IEEE

  8. Bengio Y, LeCun Y et al (2007) Scaling learning algorithms towards ai. Large-scale kernel machines 34(5):1–41

    Google Scholar 

  9. Wang P, Wang J, Li Y, Li P, Li L, Jiang M (2021) Automatic classification of breast cancer histopathological images based on deep feature fusion and enhanced routing. Biomedical Signal Processing and Control 65:102341

    Article  Google Scholar 

  10. Gupta V, Vasudev M, Doegar A, Sambyal N (2021) Breast cancer detection from histopathology images using modified residual neural networks. Biocybernetics and Biomedical Engineering 41(4):1272–1287

    Article  Google Scholar 

  11. Boumaraf S, Liu X, Zheng Z, Ma X, Ferkous C (2021) A new transfer learning based approach to magnification dependent and independent classification of breast cancer in histopathological images. Biomedical Signal Processing and Control 63:102192

    Article  Google Scholar 

  12. Bhausaheb, D.P., Kashyap, K.L.: Detection and classification of breast cancer availing deep canid optimization based deep cnn. Multimedia Tools and Applications, 1–19 (2022)

  13. Zheng Y, Jiang Z, Zhang H, Xie F, Shi J, Xue C (2019) Adaptive color deconvolution for histological wsi normalization. Computer methods and programs in biomedicine 170:107–120

    Article  PubMed  Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  15. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  16. Xie J, Liu R, Luttrell J IV, Zhang C (2019) Deep learning based analysis of histopathological images of breast cancer. Frontiers in genetics 10:80

    Article  PubMed  PubMed Central  Google Scholar 

  17. Prewitt JM, Mendelsohn ML (1966) The analysis of cell images. Annals of the New York Academy of Sciences 128(3):1035–1053

    Article  ADS  CAS  PubMed  Google Scholar 

  18. McCann MT, Ozolek JA, Castro CA, Parvin B, Kovacevic J (2014) Automated histology analysis: Opportunities for signal processing. IEEE Signal Processing Magazine 32(1):78–87

    Article  ADS  Google Scholar 

  19. Zheng Y, Jiang Z, Zhang H, Xie F, Ma Y, Shi H, Zhao Y (2018) Histopathological whole slide image analysis using context-based cbir. IEEE transactions on medical imaging 37(7):1641–1652

    Article  PubMed  Google Scholar 

  20. Komura D, Ishikawa S (2018) Machine learning methods for histopathological image analysis. Computational and structural biotechnology journal 16:34–42

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Computer graphics and applications 21(5):34–41

    Article  Google Scholar 

  22. Vahadane, A., Peng, T., Albarqouni, S., Baust, M., Steiger, K., Schlitter, A.M., Sethi, A., Esposito, I., Navab, N.: Structure-preserved color normalization for histological images. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 1012–1015 (2015). IEEE

  23. Wang, Y.-Y., Chang, S.-C., Wu, L.-W., Tsai, S.-T., Sun, Y.-N.: A color-based approach for automated segmentation in tumor tissue classification. In: 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6576–6579 (2007). IEEE

  24. Janowczyk A, Basavanhally A, Madabhushi A (2017) Stain normalization using sparse autoencoders (stanosa): application to digital pathology. Computerized Medical Imaging and Graphics 57:50–61

    Article  PubMed  Google Scholar 

  25. Bejnordi BE, Balkenhol M, Litjens G, Holland R, Bult P, Karssemeijer N, Van Der Laak JA (2016) Automated detection of dcis in whole-slide h &e stained breast histopathology images. IEEE transactions on medical imaging 35(9):2141–2150

    Article  Google Scholar 

  26. Magee, D., Treanor, D., Crellin, D., Shires, M., Smith, K., Mohee, K., Quirke, P.: Colour normalisation in digital histopathology images. In: Proc Optical Tissue Image Analysis in Microscopy, Histopathology and Endoscopy (MICCAI Workshop), vol. 100, pp. 100–111 (2009). Daniel Elson London

  27. Khan AM, Rajpoot N, Treanor D, Magee D (2014) A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE transactions on Biomedical Engineering 61(6):1729–1738

    Article  PubMed  Google Scholar 

  28. Bianconi, F., Kather, J.N., Reyes-Aldasoro, C.C.: Evaluation of colour pre-processing on patch-based classification of h &e-stained images. In: Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, Proceedings 15, pp. 56–64 (2019). Springer

  29. Ruifrok AC, Johnston DA et al (2001) Quantification of histochemical staining by color deconvolution. Analytical and quantitative cytology and histology 23(4):291–299

    CAS  PubMed  Google Scholar 

  30. Xu Y, Jia Z, Wang L-B, Ai Y, Zhang F, Lai M, Chang EI et al (2017) Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC bioinformatics 18(1):1–17

    Article  CAS  Google Scholar 

  31. Dif N, Attaoui MO, Elberrichi Z, Lebbah M, Azzag H (2022) Transfer learning from synthetic labels for histopathological images classification. Applied Intelligence 52(1):358–377

    Article  Google Scholar 

  32. Toğaçar M, Özkurt KB, Ergen B, Cömert Z (2020) Breastnet: A novel convolutional neural network model through histopathological images for the diagnosis of breast cancer. Physica A: Statistical Mechanics and its Applications 545:123592

    Article  Google Scholar 

  33. Singh R, Bharti V, Purohit V, Kumar A, Singh AK, Singh SK (2021) Metamed: Few-shot medical image classification using gradient-based meta-learning. Pattern Recognition 120:108111

    Article  Google Scholar 

  34. Zhou F, Yang S, Fujita H, Chen D, Wen C (2020) Deep learning fault diagnosis method based on global optimization gan for unbalanced data. Knowledge-Based Systems 187:104837

    Article  Google Scholar 

  35. Chattopadhyay S, Dey A, Singh PK, Oliva D, Cuevas E, Sarkar R (2022) Mtrre-net: A deep learning model for detection of breast cancer from histopathological images. Computers in Biology and Medicine 150:106155

    Article  PubMed  Google Scholar 

  36. Senousy Z, Abdelsamea MM, Gaber MM, Abdar M, Acharya UR, Khosravi A, Nahavandi S (2021) Mcua: Multi-level context and uncertainty aware dynamic deep ensemble for breast cancer histology image classification. IEEE Transactions on Biomedical Engineering 69(2):818–829

    Article  Google Scholar 

  37. Yang Z, Ran L, Zhang S, Xia Y, Zhang Y (2019) Ems-net: Ensemble of multiscale convolutional neural networks for classification of breast cancer histology images. Neurocomputing 366:46–53

    Article  Google Scholar 

  38. Sharma S, Mehra R, Kumar S (2021) Optimised cnn in conjunction with efficient pooling strategy for the multi-classification of breast cancer. IET Image Processing 15(4):936–946

    Article  Google Scholar 

  39. Senousy Z, Abdelsamea MM, Mohamed MM, Gaber MM (2021) 3e-net: Entropy-based elastic ensemble of deep convolutional neural networks for grading of invasive breast carcinoma histopathological microscopic images. Entropy 23(5):620

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  40. Kallipolitis A, Revelos K, Maglogiannis I (2021) Ensembling efficientnets for the classification and interpretation of histopathology images. Algorithms 14(10):278

    Article  Google Scholar 

  41. Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M (2022) Attention mechanisms in computer vision: A survey. Computational Visual Media 8(3):331–368

    Article  Google Scholar 

  42. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  43. Mnih, V., Heess, N., Graves, A., et al.: Recurrent models of visual attention. Advances in neural information processing systems 27 (2014)

  44. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. Advances in neural information processing systems 28 (2015)

  45. Hu, J., Shen, L., Albanie, S., Sun, G., Vedaldi, A.: Gather-excite: Exploiting feature context in convolutional neural networks. Advances in neural information processing systems 31 (2018)

  46. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  47. Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., Tang, X.: Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2017)

  48. Kather, J.N., Halama, N., Marx, A.: 100,000 histological images of human colorectal cancer and healthy tissue. Zenodo10 5281 (2018)

  49. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2015) A dataset for breast cancer histopathological image classification. Ieee transactions on biomedical engineering 63(7):1455–1462

    Article  PubMed  Google Scholar 

  50. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)

  51. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  52. Ghosh S, Bandyopadhyay A, Sahay S, Ghosh R, Kundu I, Santosh K (2021) Colorectal histology tumor detection using ensemble deep neural network. Engineering Applications of Artificial Intelligence 100:104202

    Article  Google Scholar 

  53. Kumar A, Vishwakarma A, Bajaj V (2023) Crccn-net: Automated framework for classification of colorectal tissue using histopathological images. Biomedical Signal Processing and Control 79:104172

    Article  Google Scholar 

  54. Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2021)

  55. Litjens G, Bandi P, Ehteshami Bejnordi B, Geessink O, Balkenhol M, Bult P, Halilovic A, Hermsen M, van de Loo R, Vogels R et al (2018) 1399 h &e-stained sentinel lymph node sections of breast cancer patients: the camelyon dataset. GigaScience 7(6):065

    Article  Google Scholar 

Download references

Funding

This work was supported by the Natural Science Foundation of Heilongjiang Province of China (No. JJ2019JQ0013)

Author information

Authors and Affiliations

  1. College of Mecheanical and Electrical Engineering, Northeast Forestry University, Harbin, 150040, China

    Yining Xie & Yuming Zhang

  2. School of Computer Science and Technology, Harbin University of Science and Technology, Harbin, 150080, China

    Jianxin Hou & Deyun Chen

  3. China Airlines Harbin Dongan Engine Co.,Ltd, Harbin, 150066, China

    Guohui Guan

Authors
  1. Yining Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianxin Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Deyun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guohui Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by Yining Xie, Yuming Zhang, Jianxin Hou, and Deyun Chen. The first draft of the manuscript was written by Yining Xie, and all authors commented on previous versions. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yining Xie.

Ethics declarations

Financial Interests

The authors declare they have no financial interests.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Y., Zhang, Y., Hou, J. et al. SSANet: spatial stain attention network for pathological images classification. Multimed Tools Appl 83, 33489–33510 (2024). https://doi.org/10.1007/s11042-023-16313-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16313-w

Keywords

Search

Navigation