Deep learning based feature representation for automated skin histopathological image annotation | Multimedia Tools and Applications Skip to main content

Advertisement

Springer Nature Link
Log in

Deep learning based feature representation for automated skin histopathological image annotation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automated annotation of skin biopsy histopathological images provides valuable information and supports for diagnosis, especially for the discrimination between malignant and benign lesions. Currently, computer-aid analysis of skin biopsy images mostly relied on some human-designed features, which requires expensive human efforts and experiences in problem domains. In this study, we propose an annotation framework for automated skin biopsy image analysis which makes use of a deep model for image feature representation. A convolutional neural network (CNN) is designed for local regions of skin biopsy images which learns potential high-level features automatically from input raw pixels. The annotation model is constructed in the multiple-instance multiple-label (MIML) learning framework with the features learned through the network. We achieve significant improvement of the model performance on a real world clinical skin biopsy image dataset and a benchmark dataset. Moreover, our study indicates that deep learning based model could achieve better performance than human designed features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ali R, Gunduz-Demir C, Szilágyi T, Durkee B, Graves EE (2013) Semi-automatic segmentation of subcutaneous tumours from micro-computed tomography images. Phys Med Biol 58(22):8007

    Article  Google Scholar 

  2. Angel CR, Juan CC, Fabio AG (2011) Visual pattern mining in histology image collections using bag of features. Journal Artificial Intelligence in Medicine 52(2011):91–106

    Google Scholar 

  3. Baldi A, Murace R, Dragonetti E, Manganaro M, Bizzi S (2014) Automated content-based image retrieval: Application on dermoscopic images of pigmented skin lesions. In: Skin Cancer. Springer, pp 523–528

  4. Bengio Y (2009) Learning deep architectures for ai. Found Trends Mach Learn 2(1):1–127

    Article  MATH  Google Scholar 

  5. Bengio Y (2013) Deep learning of representations: Looking forward. In: Proceedings of the 1st International Conference on Statistical Language and Speech Processing, Springer-Verlag, Berlin, Heidelberg, SLSP’13, pp 1–37

  6. Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer-Verlag, Secaucus

    MATH  Google Scholar 

  7. Bunte K, Biehl M, Jonkman MF, Petkov N (2011) Learning effective color features for content based image retrieval in dermatology. Pattern Recogn 44(9):1892–1902

    Article  Google Scholar 

  8. Caicedo JC, Cruz-Roa A, González FA (2009) Histopathology image classification using bag of features and kernel functions. In: Combi C, Shahar Y, Abu-Hanna A (eds) Proceedings of AIME, Lecture Notes in Computer Science, vol 5651, pp 126–135

  9. Cerroni L, Argenyi Z, Cerio R, Facchetti F, Kittler H, Kutzner H, Requena L, Sangueza OP, Smoller B, Wechsler J, Kerl H (2010) Influence of evaluation of clinical pictures on the histopathologic diagnosis of inflammatory skin disorders. J Am Acad Dermatol 63(4):647–52

    Article  Google Scholar 

  10. Chen J, Zhao F, Cao H (2013) Knowledge acquisition from generalized experts oriented to product innovation. In: 2013 6th international conference on Information management, innovation management and industrial engineering (ICIII), vol 2. IEEE, pp 546–549

  11. Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5:913–939

    MathSciNet  Google Scholar 

  12. Cho Y (2012) Kernel methods for deep learning. PhD thesis, La Jolla, CA, USA, aAI3513249

  13. Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89(1-2):31–71

    Article  MATH  Google Scholar 

  14. Dong R (2011) Feature grouping technique to relax sample support requirement for sparse linear feature extraction. In: 2011 seventh international conference on Natural computation (ICNC), vol 3. IEEE, pp 1654–1657

  15. Ferrara G, Argenyi Z, Argenziano G, Cerio R, Cerroni L, Di Blasi A, Feudale EA, Giorgio CM, Massone C, Nappi O, Tomasini C, Urso C, Zalaudek I, Kittler H, Soyer HP (2009) The influence of clinical information in the histopathologic diagnosis of melanocytic skin neoplasms. PLoS One 4(4):e5375

    Article  Google Scholar 

  16. Foulds J, Frank E (2010) A review of multi-instance learning assumptions. Knowl Eng Rev 25(1):1–25

    Article  Google Scholar 

  17. Gao L, Song J, Zou F, Zhang D, Shao J (2015) Scalable multimedia retrieval by deep learning hashing with relative similarity learning. In: Proceedings of the 23rd ACM International Conference on Multimedia, ACM, New York, NY, USA, MM ’15, pp 903–906

  18. He J, Gu H, Wang Z (2012) Bayesian multi-instance multi-label learning using gaussian process prior. Mach Learn 88(1-2):273–295

    Article  MathSciNet  MATH  Google Scholar 

  19. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  20. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–54

    Article  MathSciNet  MATH  Google Scholar 

  21. Huang PS, He X, Gao J, Deng L, Acero A, Heck L (2013) Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Conference on Information & Knowledge Management, ACM, New York, NY, USA, CIKM ’13, pp 2333–2338

  22. Li Y, Su H, Qi CR, Fish N, Cohen-Or D, Guibas LJ (2015) Joint embeddings of shapes and images via cnn image purification. ACM Trans Graph 34(6):234:1–234:12

    Google Scholar 

  23. Lopes N, Ribeiro B (2014) Towards adaptive learning with improved convergence of deep belief networks on graphics processing units. Pattern Recogn 47(1):114–127

    Article  Google Scholar 

  24. Malik MSA, Sulaiman S (2014) Dba’s perspective on use of information visualization in electronic health records

  25. Marrugo AG, Millan MS (2011) Retinal image analysis: preprocessing and feature extraction. Journal of Physics: Conference Series, IOP Publishing 274:012039

    Google Scholar 

  26. Murthy VN, Maji S, Manmatha R (2015) Automatic image annotation using deep learning representations. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, ACM, New York, NY, USA, ICMR ’15, pp 603–606

  27. Ooi BC, Tan KL, Wang S, Wang W, Cai Q, Chen G, Gao J, Luo Z, Tung AK, Wang Y, Xie Z, Zhang M, Zheng K (2015) Singa: A distributed deep learning platform. In: Proceedings of the 23rd ACM International Conference on Multimedia, ACM, New York, NY, USA, MM ’15, pp 685–688

  28. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2-3):135–168

    Article  MATH  Google Scholar 

  29. Shi J, Malik J (2000) NorMalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  30. Stoean C, Stoean R, Sandita A, Mesina C, Gruia CL, Ciobanu D (2015) Evolutionary search for an accurate contour segmentation in histopathological images. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, ACM, New York, NY, USA, GECCO Companion ’15, pp 1491–1492

  31. Tang Y (2013) Deep learning using support vector machines. CoRR arXiv:1306.0239

  32. Tipping ME (2001) Sparse bayesian learning and the relevance vector machine. J Mach Learn Res 1:211–244

    MathSciNet  MATH  Google Scholar 

  33. Trinh HP, Duranton M, Paindavoine M (2015) Efficient data encoding for convolutional neural network application. ACM Trans Archit Code Optim 11(4):49:1–49:21

    Article  Google Scholar 

  34. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion 11:3371–3408

  35. Zhang G, Shu X, Liang Z, Liang Y, Chen S, Yin J (2012a) Multi-instance learning for skin biopsy image features recognition, Philadelphia, PA, United states, pp 83–88

  36. Zhang G, Yin J, Li Z, Liang Z, Fu W (2012b) Deep learning for acupuncture point selection patterns based on veteran doctor experience of chinese medicine. In: Proceedings of the 2012 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW), IEEE Computer Society, Washington, DC, USA, BIBMW ’12, pp 396–401

  37. Zhang G, Yin J, Li Z, Su X, Li G, Zhang H (2013) Automated skin biopsy histopathological image annotation using multi-instance representation and learning. BMC Med Genet 6(3):1–14

    Google Scholar 

  38. Zhang G, Yin J, Su XY, Huang YJ, Lao YR, Liang ZH, Ou SX, Zhang HL (2014) Augmenting multi-instance multilabel learning with sparse bayesian models for skin biopsy image analysis. BioMed Research International 2014(Article ID 305629):13 pages

  39. Zhang ML (2009) Generalized multi-instance learning: Problems, algorithms and data sets

  40. Zhang S, Huang Q, Hua G, Jiang S, Gao W, Tian Q (2010) Building contextual visual vocabulary for large-scale image applications. In: Proceedings of the international conference on Multimedia. ACM, pp 501–510

  41. Zhong R, Tezuka T (2014) Parametric learning of deep convolutional neural network. In: Proceedings of the 19th International Database Engineering & Applications Symposium, ACM, New York, NY, USA, IDEAS ’15, pp 226–227

  42. Zhou ZH, Zhang ML, Huang SJ, Li YF (2012) Multi-instance multi-label learning. Artif Intell 176(1):2291–2320

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 61502106, 81373883, 81573827), Natural Science Foundation of GuangDong Province (No. 2016A030310340), the College Student Career and Innovation Training Plan Project of Guangdong Province (xj201511845018, yj201511845038, yj201611845074, yj201611845075, yj201611845366), the Special Fund of Cultivation of Technology Innovation for University Students (pdjh2016b0150), the 2015 Research Project of Guangdong Education Evaluation Association (No. G-11) and Fujian Major Project of Regional Industry (No. 2014H4015).

Author information

Authors and Affiliations

  1. School of Automation, Guangdong University of Technology, Guangzhou, China

    Gang Zhang & Huadong Lai

  2. Department of Computer Science and Information Engineering, Chung Hua University, Hsinchu, 300, Taiwan

    Ching-Hsien Robert Hsu

  3. College of Mathematics and Computer Science, Fuzhou University, Fuzhou, China

    Xianghan Zheng

Authors
  1. Gang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Hsien Robert Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huadong Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianghan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianghan Zheng.

Ethics declarations

Competing Interests

The authors declare that there is no conflict of interests regarding the publication of this article.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, G., Hsu, CH.R., Lai, H. et al. Deep learning based feature representation for automated skin histopathological image annotation. Multimed Tools Appl 77, 9849–9869 (2018). https://doi.org/10.1007/s11042-017-4788-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4788-5

Keywords

Search

Navigation