Triple-View Feature Learning for Medical Image Segmentation

Abstract

Deep learning models, e.g. supervised Encoder-Decoder style networks, exhibit promising performance in medical image segmentation, but come with a high labelling cost. We propose TriSegNet, a semi-supervised semantic segmentation framework. It uses triple-view feature learning on a limited amount of labelled data and a large amount of unlabeled data. The triple-view architecture consists of three pixel-level classifiers and a low-level shared-weight learning module. The model is first initialized with labelled data. Label processing, including data perturbation, confidence label voting and unconfident label detection for annotation, enables the model to train on labelled and unlabeled data simultaneously. The confidence of each model gets improved through the other two views of the feature learning. This process is repeated until each model reaches the same confidence level as its counterparts. This strategy enables triple-view learning of generic medical image datasets. Bespoke overlap-based and boundary-based loss functions are tailored to the different stages of the training. The segmentation results are evaluated on four publicly available benchmark datasets including Ultrasound, CT, MRI, and Histology images. Repeated experiments demonstrate the effectiveness of the proposed network compared against other semi-supervised algorithms, across a large set of evaluation measures.

Appendices

A Algorithm of TriSegNet

The training of TriSegNet consists of four stages which is briefly illustrated in Algorithm 1. The code of TriSegNet will be publicly available².

B The CNN Architecture of Multi-view Learning

To properly encourage the differences of the three views of feature learning on dense prediction, not only the data feed and initialization of parameters, but three different advanced CNN are proposed in TriSegNet. We utilize three different techniques for CNN i.e. skip connection, efficiently passing feature information through residual learning, and multi-scale feature learning. The parameters of three classifiers are briefly illustrated in Table 4 and the source code has been released online³.

Table 3. The computation cost information of three classifier

C Evaluation Methods, Qualitative, and Quantitative Results

Table 2 reports the TriSegNet performance direct comparison with other algorithms with several strict and novel quantitative evaluation metrics to which the boundaries of the machine segmentation(MS) match those of the ground truth(GT), using the Directed Boundary Dice relative to GT (DBD$_G$), Directed Boundary Dice relative to MS (DBD$_M$) and Symmetric Boundary Dice (SBD).

In a von Neumann neighbourhood $N_x$ of each pixel x on the boundary $\partial G$ of the ground truth,

$$\begin{aligned} DBD_G=DBD(G, M)=\displaystyle \frac{\sum \limits _{x \in \partial G} \text {Dice}(N_x)}{\left| \partial G \right| } \end{aligned}$$

(4)

$$\begin{aligned} DBD_M=DBD(M, G)=\displaystyle \frac{\sum \limits _{x \in \partial M} \text {Dice}(N_Y)}{\left| \partial M \right| } \end{aligned}$$

(5)

$$\begin{aligned} SBD = \displaystyle \frac{\sum \limits _{x \in \partial G} DSC(N_x) + \sum \limits _{y \in \partial M} DSC(N_y)}{\left| \partial G \right| + \left| \partial M \right| } \end{aligned}$$

(6)

where Dice is $Dice(N_x) = \frac{2 | G(N_x) \cap M(N_y)|}{| G(N_x)| + | M(N_y)|}$. The symmetric average is being brought down by DBD$_G$ when the latter features isolated areas of false negative labels. These measures penalise mislabelled areas in the machine segmentation.

Some of example qualitative results on MRI Cardiac test set are briefly sketched in Fig. 4. Eight images are selected from MRI test set where the first row illustrates raw images. The rest of them illustrate the MS by each semi-supervised algorithm against GT where yellow, green, red, and black represent true positive, false negative, false positive and true negative at pixel level. The proposed method shows fewer false positive and false negative pixels, and significantly low HD as well, because the TriSegNet is beneficial with different views of high-level pixel-level classifier and proposed mixed boundary- and overlap-based loss function.

Fig. 4.

Sample qualitative results on MRI cardiac test set