Hybrid descriptor for placental maturity grading | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

Hybrid descriptor for placental maturity grading

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Placental maturity grading (PMG) is quite essential to assess fetal growth and maternal health. To this date, PMG has mostly relied on the subjective judgment of the clinician, which is time-consuming and may cause wrong estimation due to redundancy and repeatability of the process. To tackle it, we propose an automatic method to stage placental maturity via deep hybrid descriptors based on B-mode ultrasound (BUS) and color Doppler energy (CDE) images. Specifically, convolutional descriptors extracted from multiple deep convolutional neural networks (DCNNs) and hand-crafted features are integrated to get the hybrid descriptors for grading performance boosting. First, different models with various feature layers are combined to obtain hybrid descriptors from images. Second, the transfer learning strategy is also utilized to enhance the grading performance via the deeply represented features. Third, extracted descriptors are encoded by Fisher vector (FV). Finally, we use support vector machine (SVM) as the classifier to grade placental maturity. The experimental results demonstrate that our proposed method could achieve good performance in PMG.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bude RO, Rubin JM (1996) Power Doppler sonography. Radiology 200(1):21–23

    Google Scholar 

  2. Burton GJ, Charnock-Jones DS, Jauniaux E (2009) Regulation of vascular growth and function in the human placenta. Reproduction 138(6):895–902

    Google Scholar 

  3. Chang H, Chen Z, Huang Q, Shi J, Li X (2015) Graph-based learning for segmentation of 3D ultrasound images. Neurocomputing 151:632–644

    Google Scholar 

  4. Chen H, Ni D, Qin J, Li S, Yang X, Wang T, Heng PA (2015) Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J Biomed Health Inform 19(5):1627–1636

    Google Scholar 

  5. Chen H, Wu L, Dou Q, Qin J, Li S, Cheng J-Z, Ni D, Heng P-A (2017) Ultrasound standard plane detection using a composite neural network framework. IEEE Transactions on Cybernetics 47(6):1576–1586

    Google Scholar 

  6. Cheng J-Z, Ni D, Chou Y-H, Qin J, Tiu C-M, Chang Y-C, Huang C-S, Shen D, Chen C-M (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in us images and pulmonary nodules in CT scans. Sci Rep 6:24454

    Google Scholar 

  7. Cimpoi M, Maji S, Vedaldi A (2015) A Deep filter banks for texture recognition and segmentation. In: Proc IEEE CVPR: pp 3828–3836

  8. Cui C, Liu H, Lian T, Nie L, Zhu L, Yin Y (2018) Distribution-oriented aesthetics assessment with semantic-aware hybrid network. IEEE Transactions on Multimedia 21(5):1209–1220

    Google Scholar 

  9. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J Royal Statistical Soc 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  10. D'Hooge J, Heimdal A, Jamal F, Kukulski T, Bijnens B, Rademakers F, Hatle L, Suetens P, Sutherland GR (2000) Regional strain and strain rate measurements by cardiac ultrasound: principles, implementation and limitations. Eur Heart J Cardiovasc Imaging 1(3):154–170

    Google Scholar 

  11. Dubiel M, Breborowicz GH, Ropacka M, Pietryga M, Maulik D, Gudmundsson S (2005) Computer analysis of three-dimensional power angiography images of foetal cerebral, lung and placental circulation in normal and high-risk pregnancy. Ultrasound Med Biol 31(3):321–327

    Google Scholar 

  12. Elsayes KM, Trout AT, Friedkin AM, Liu PS, Bude RO, Platt JF, Menias CO (2009) Imaging of the placenta: a multimodality pictorial review. RadioGraphics 29(5):1371–1391

    Google Scholar 

  13. Faraki M, Harandi MT, Wiliem A, Lovell BC (2014) Fisher tensors for classifying human epithelial cells. Pattern Recogn 47(7):2348–2359

    Google Scholar 

  14. Goldenberg RL, Gravett MG, Iams J, Papageorghiou AT, Waller SA, Kramer M, Culhane J, Barros F, Conde-Agudelo A, Bhutta ZA, Knight HE, Villar J (2012) The preterm birth syndrome: issues to consider in creating a classification system. Am J Obstet Gynecol 206(2):113–118

    Google Scholar 

  15. Gong Y, Wang L, Guo R, Lazebnik S (2014) Multi-scale orderless pooling of deep convolutional activation features. In: Proc ECCV: pp 392-407

  16. Grannum PAT, Berkowitz RL, Hobbins JC (1979) The ultrasonic changes in the maturing placenta and their relation to fetal pulmonic maturity. Am J Obstet Gynecol 133(8):915–922

    Google Scholar 

  17. Guerriero S, Ajossa S, Lai MP, Risalvato A, Paoletti AM, Melis GB (1999) Clinical applications of colour Doppler energy imaging in the female reproductive tract and pregnancy. Hum Reprod Update 5(5):515–529

    Google Scholar 

  18. Guiot C, Gaglioti P, Oberto M, Piccoli E, Rosato R, Todros T (2008) Is three-dimensional power Doppler ultrasound useful in the assessment of placental perfusion in normal and growth-restricted pregnancies? Ultrasound Obstet Gynecol 31(2):171–176

    Google Scholar 

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE CVPR: pp 770-778

  20. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: proceedings of the IEEE conference on computer vision and pattern recognition: pp 7132-7141

  21. Huang Q, Huang Y, Hu W, Li X (2015) Bezier interpolation for 3-D freehand ultrasound. IEEE Trans Hum Mach Syst 45(3):385–392

    Google Scholar 

  22. Huang Q, Xie B, Ye P, Chen Z (2015) Correspondence - 3-D ultrasonic strain imaging based on a linear scanning system. IEEE Trans Ultrason Ferroelectr Freq Control 62(2):392–400

    Google Scholar 

  23. Jeff D, Yangqing J, Oriol V, Judy H, Ning Z, Eric T, Trevor D (2014) DeCAF: a deep convolutional activation feature for generic visual recognition. In: Proc ICML: pp 647-655

  24. Jégou H, Perronnin F, Douze M, Sánchez J, Pérez P, Schmid C (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716

    Google Scholar 

  25. Kazzi GM, Gross TL, Sokol RJ, Kazzi NJ (1983) Detection of intrauterine growth retardation: a new use for sonographic placental grading. Am J Obstet Gynecol 145(6):733–737

    Google Scholar 

  26. Kellow ZS, Feldstein VA (2011) Ultrasound of the placenta and umbilical cord: a review. Ultrasound Q 27(3):187–197

    Google Scholar 

  27. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proc Adv NIPS: pp 1097-1105

  28. Lazebnik S, Schmid C (2006) Ponce J beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proc IEEE CVPR. pp 2169–2178

  29. Lei B, Li W, Yao Y, Jiang X, Tan E-L, Qin J, Chen S, Ni D, Wang T (2017) Multi-modal and multi-layout discriminative learning for placental maturity staging. Pattern Recogn 63:719–730

    Google Scholar 

  30. Lei B, Li X, Yao Y, Li S, Chen S, Zhou Y, Ni D, Wang T (2014) Automatic grading of placental maturity based on LIOP and fisher vector. In: Proc IEEE EMBC, Chicago, pp 4671–4674

  31. Lei B, Tan E-L, Chen S, Li W, Ni D, Yao Y, Wang T (2017) Automatic placental maturity grading via hybrid learning. Neurocomputing 223:86–102

    Google Scholar 

  32. Lei B, Tan E-L, Chen S, Zhuo L, Li S, Ni D, Wang T (2015) Automatic recognition of fetal facial standard plane in ultrasound image via fisher vector. PLoS One 10(5):e0121838

    Google Scholar 

  33. Li W, Yao Y, Ni D, Chen S, Lei B, Wang T (2016) Placental maturity evaluation via feature fusion and discriminative learning. In: Proc IEEE ISBI: pp 783-786

  34. Li X, Yao Y, Ni D, Chen S, Li S, Lei B, Wang T (2014) Automatic staging of placental maturity based on dense descriptor. Biomed Mater Eng 24(6):2821–2829

    Google Scholar 

  35. Linares PA, McCullagh PJ, Black ND (2004) Dornan J feature selection for the characterization of ultrasonic images of the placenta using texture classification. In: Proc IEEE ISBI. pp 1147–1150

  36. Liu L, Shen C, Wang L, Hengel AVD, Wang C (2014) Encoding high dimensional local features by sparse coding based fisher vectors. In: Proc Adv NIPS: pp 1143-1151

  37. Liu Z, Zheng H, Lin S (2009) Application of multi-classification support vector machine in the B-placenta image classification. In: IEEE Int Conf Comput Intell SW Eng: pp 1-4

  38. Long J, Zhang N, Darrell T (2014) do convnets learn correspondence? In: Proc Adv NIPS: pp 1601-1609

  39. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Google Scholar 

  40. Moran M, Mulcahy C, Daly L, Zombori G, Downey P, McAuliffe FM (2014) Novel placental ultrasound assessment: potential role in pre-gestational diabetic pregnancy. Placenta 35(8):639–644

    Google Scholar 

  41. Mousavian A, Kosecka J (2015) Deep convolutional features for image based retrieval and scene categorization. arXiv:150906033

  42. Ng YH, Yang F, Davis LS (2015) Exploiting local features from deep networks for image retrieval. Proc IEEE CVPR:53–61

  43. Ozcan T, Pressman EK (2008) Imaging of the placenta. Ultrasound Clin 3(1):13–22

    Google Scholar 

  44. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Transactions on Knowledge Data Engineering 22(10):1345–1359

    Google Scholar 

  45. Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proc IEEE CVPR: pp 806-813

  46. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137

    Google Scholar 

  47. Reynolds D (2009) Gaussian mixture models. In: Encyclopedia of Biometrics. Springer, pp 93–105

  48. Ribeiro RT, Marinho RT, Sanches JM (2013) Classification and staging of chronic liver disease from multimodal data. IEEE Trans Biomed Eng 60(5):1336–1344

    Google Scholar 

  49. Sánchez J, Perronnin F, Mensink T, Verbeek J (2013) Image classification with the fisher vector: theory and practice. Int J Comput Vis 105(3):222–245

    MathSciNet  MATH  Google Scholar 

  50. Sergey I, Christian S (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proc PMLR: pp 448-456

  51. Shi J, Xue Z, Dai Y, Peng B, Dong Y, Zhang Q, Zhang Y (2018) Cascaded multi-column RVFL+ classifier for single-modal neuroimaging-based diagnosis of Parkinson's disease. IEEE Trans Biomed Eng 66(8):2362–2371

    Google Scholar 

  52. Simonyan K, Zisserman A (2014) A very deep convolutional networks for large-scale image recognition. In: Proc IEEE CVPR

  53. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Proc Adv NIPS: pp 3320-3328

  54. Zheng L, Zhao Y, Wang S, Wang J, Tian Q (2016) Good practice in CNN feature transfer. arXiv:160400133

  55. Zheng W, Zhu X, Wen G, Zhu Y, Yu H, Gan J (2018) Unsupervised feature selection by self-paced learning regularization. Pattern Recogn Lett

Download references

Acknowledgments

This work was supported partly by National Key R&D Program of China (No.2016YFC0104700), National Natural Science Foundation of China (Nos.61871274, 61801305 and 81571758), National Natural Science Foundation of Guangdong Province (No. 2017A030313377), Guangdong Pearl River Talents Plan (2016ZT06S220), Medical Scientific Research Foundation of Guangdong Province, China (No. B2018031), Shenzhen Peacock Plan (Nos. KQTD2016053112051497 and KQTD2015033016 104926), and Shenzhen Key Basic Research Project (Nos. JCYJ20180507184647636, JCYJ20170818142347251 and JCYJ20170818 094109846).

Author information

Authors and Affiliations

  1. School of Biomedical Engineering, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Shenzhen University, Shenzhen, China

    Baiying Lei, Feng Jiang, Dong Ni, Siping Chen & Tianfu Wang

  2. Department of Industrial and Manufacturing, Systems Engineering, The University of Michigan, Dearborn, MI, USA

    Feng Zhou

  3. Department of Ultrasound, Shenzhen Maternal and Child Healthcare Hospital, 3012 Fuqiang Rd, Shenzhen, China

    Yuan Yao

Authors
  1. Baiying Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dong Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuan Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Siping Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tianfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianfu Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, B., Jiang, F., Zhou, F. et al. Hybrid descriptor for placental maturity grading. Multimed Tools Appl 79, 21223–21239 (2020). https://doi.org/10.1007/s11042-019-08489-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08489-x

Keywords

Search

Navigation