Deep Learning in Medical Image Analysis - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2017 Jun 21:19:221-248.
doi: 10.1146/annurev-bioeng-071516-044442. Epub 2017 Mar 9.

Deep Learning in Medical Image Analysis

Dinggang Shen  1   2 , Guorong Wu  1 , Heung-Il Suk  2
Affiliations
Review

Deep Learning in Medical Image Analysis

Dinggang Shen et al. Annu Rev Biomed Eng. .

Abstract

This review covers computer-assisted analysis of images in the field of medical imaging. Recent advances in machine learning, especially with regard to deep learning, are helping to identify, classify, and quantify patterns in medical images. At the core of these advances is the ability to exploit hierarchical feature representations learned solely from data, instead of features designed by hand according to domain-specific knowledge. Deep learning is rapidly becoming the state of the art, leading to enhanced performance in various medical applications. We introduce the fundamentals of deep learning methods and review their successes in image registration, detection of anatomical and cellular structures, tissue segmentation, computer-aided disease diagnosis and prognosis, and so on. We conclude by discussing research issues and suggesting future directions for further improvement.

Keywords: deep learning; medical image analysis; unsupervised feature learning.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Architectures of feed-forward neural networks.
Figure 2
Figure 2
Three representative deep models with vectorized inputs for unsupervised feature learning. The red links, whether they are directed or undirected, denote the full connections of units in two consecutive layers but no connections among units in the same layer. Note the differences among models in directed/undirected connections and directions of connections that depict the conditional relationships.
Figure 3
Figure 3
A graphical illustration of three key mechanisms (i.e., local receptive field, weights sharing, and subsampling) in convolutional neural networks.
Figure 4
Figure 4
Construction of a deep encoder-decoder via SAE and visualization of the learned feature representations.
Figure 5
Figure 5
The similarity maps of identifying the correspondence for the red-crossed point in the template (a) w.r.t. the subject (b) by handcraft features (d–e) and the SAE learned features by unsupervised deep learning (f). The registered subject image is shown in (c). It is clear that the inaccurate registration results might undermine the supervised feature representation learning that highly relies on the correspondences across all training images.
Figure 6
Figure 6
Typical registration results on 7.0-Tesla MR brain images by Demons (83), HAMMER (84), and HAMMER combined with SAE-learned feature representations, respectively. Three rows represent three different slices in the template, subject, and registered subjects.
Figure 7
Figure 7
Typical prostate segmentation results of two different patients produced by three different feature representations. Red contours indicate the manual ground-truth segmentations, and yellow contours indicate the automatic segmentations. The second and fourth rows show the 3D visualization of the segmentation results corresponding to the images above. For each 3D visualization, the red surfaces indicate the automatic segmentation results using different features, such as intensity, handcrafted, and deep learning, respectively. The transparent grey surfaces indicate the ground-truth segmentations.
Figure 8
Figure 8
An architecture of the fully convolutional network used for tissue segmentation in (44).
Figure 9
Figure 9
An illustration of (a) the shared feature learning from patches of the heterogeneous modalities, e.g., MRI and PET, with discriminative multi-modal DBM and (b, c) visualization of the learned weights in Gaussian RBMs (bottom) and those of the first hidden layer (top) from MRI and PET pathways in multi-modal DBM (25).
Figure 10
Figure 10
Functional networks learned from the first hidden layer of Suk et al.’s deep auto-encoder (29). Functional networks on the left column, from top to bottom, correspond to the default-mode network, executive attention network, visual network, subcortical regions, and cerebellum. On the right column, these show the relations among regions of different networks, cortices, and cerebellum.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Brody H. Medical imaging. Nature. 2013;502:S81–S81. - PubMed
    1. Schmidhuber J. Deep learning in neural networks: An overview. Neural Networks. 2015;61:85–117. - PubMed
    1. Bengio Y. Learning deep architectures for ai. Foundations and Trends in Machine Learning. 2009;2:1–127.
    1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444. - PubMed
    1. Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science. 2006;313:504–507. - PubMed

MeSH terms