Abstract
Many biological and medical tasks require the delineation of 3D curvilinear structures such as blood vessels and neurites from image volumes. This is typically done using neural networks trained by minimizing voxel-wise loss functions that do not capture the topological properties of these structures. As a result, the connectivity of the recovered structures is often wrong, which lessens their usefulness. In this paper, we propose to improve the 3D connectivity of our results by minimizing a sum of topology-aware losses on their 2D projections. This suffices to increase the accuracy and to reduce the annotation effort required to provide the required annotated training data.
D. Onerand and H. Osman—The two authors contributed equally to this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Breitenreicher, D., Sofka, M., Britzen, S., Zhou, S.K.: Hierarchical discriminative framework for detecting tubular structures in 3D images. In: Gee, J.C., Joshi, S., Pohl, K.M., Wells, W.M., Zöllei, L. (eds.) IPMI 2013. LNCS, vol. 7917, pp. 328–339. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38868-2_28
Briggman, K., Denk, W., Seung, S., Helmstaedter, M., Turaga, S.: Maximin affinity learning of image segmentation. In: Advances in Neural Information Processing Systems, pp. 1865–1873 (2009)
Bullitt, E., et al.: Vessel tortuosity and brain tumor malignancy: a blinded study. Acad. Radiol. 12(10), 1232–1240 (2005)
Byrne, N., Clough, J.R., Montana, G., King, A.P.: A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI. In: Puyol Anton, E., et al. (eds.) STACOM 2020. LNCS, vol. 12592, pp. 3–13. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68107-4_1
Chen, S., Ma, K., Zheng, Y.: Med3D: transfer learning for 3D medical image analysis. arXiv preprint (2019)
Clough, J., Byrne, N., Oksuz, I., Zimmer, V.A., Schnabel, J.A., King, A.: A topological loss function for deep-learning based image segmentation using persistent homology. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Edelsbrunner, H., Harer, J.: Persistent homology - a survey. Contemp. Math. 453, 257–282 (2008)
Van Etten, A., Lindenbaum, D., Bacastow, T.: Spacenet: A Remote Sensing Dataset and Challenge Series. arXiv preprint (2018)
Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056195
Funke, J., et al.: Large scale image segmentation with structured loss based deep learning for connectome reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1669–1680 (2018)
Ganin, Y., Lempitsky, V.: N4-fields: neural network nearest neighbor fields for image transforms. In: Asian Conference on Computer Vision, pp. 536–551 (2014)
Guo, Z., et al.: Deepcenterline: a multi-task fully convolutional network for centerline extraction. In: IPMI, pp. 441–453 (2019)
Hu, X., Li, F., Samaras, D., Chen, C.: Topology-preserving deep image segmentation. In: Advances in Neural Information Processing Systems, pp. 5658–5669 (2019)
Hu, X., Wang, Y., Fuxin, L., Samaras, D., Chen, C.: Topology-aware segmentation using discrete Morse theory. In: International Conference on Learning Representations (2021)
Huang, X., Zhang, L.: Road centreline extraction from high-resolution imagery based on multiscale structural features and support vector machines. Int. J. Rem. Sens. 30, 1977–1987 (2009)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimisation. In: International Conference on Learning Representations (2015)
Koziński, M., Mosińska, A., Salzmann, M., Fua, P.: Learning to segment 3D linear structures using only 2D annotations. In: Conference on Medical Image Computing and Computer Assisted Intervention, pp. 283–291 (2018)
Koziński, M., Mosinska, A., Salzmann, M., Fua, P.: Tracing in 2D to reduce the annotation effort for 3D deep delineation of linear structures. Med. Image Anal. 60 (2020)
Law, M.W.K., Chung, A.C.S.: Three dimensional curvilinear structure detection using optimally oriented flux. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 368–382. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88693-8_27
Maninis, K.K., Pont-Tuset, J., Arbeláez, P., Van Gool, L.: Deep retinal image understanding. In: Conference on Medical Image Computing and Computer Assisted Intervention, pp. 140–148 (2016)
Mnih, V., Hinton, G.E.: Learning to detect roads in high-resolution aerial images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 210–223. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15567-3_16
Mosińska, A., Kozinski, M., Fua, P.: Joint segmentation and path classification of curvilinear structures. IEEE Trans. Pattern Anal. Mach. Intell. 42(6), 1515–1521 (2020)
Mosińska, A., Marquez-Neila, P., Kozinski, M., Fua, P.: Beyond the pixel-wise loss for topology-aware delineation. In: Conference on Computer Vision and Pattern Recognition, pp. 3136–3145 (2018)
Oner, D., Koziński, M., Citraro, L., Dadap, N.C., Konings, A.G., Fua, P.: Promoting connectivity of network-like structures by enforcing region separation. IEEE Trans. Pattern Anal. Mach. Intell. (2021)
Peng, H., Zhou, Z., Meijering, E., Zhao, T., Ascoli, G.A., Hawrylycz, M.: Automatic tracing of ultra-volumes of neuronal images. Nat. Methods 14, 332–333 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Conference on Medical Image Computing and Computer Assisted Intervention, pp. 234–241 (2015)
Shit, S., et al.: cldice - a novel topology-preserving loss function for tubular structure segmentation. In: CVPR, pp. 16560–16569. Computer Vision Foundation/IEEE (2021)
Sironi, A., Lepetit, V., Fua, P.: Multiscale centerline detection by learning a scale-space distance transform. In: Conference on Computer Vision and Pattern Recognition (2014)
Turetken, E., Becker, C., Glowacki, P., Benmansour, F., Fua, P.: Detecting irregular curvilinear structures in gray scale and color imagery using multi-directional oriented flux. In: International Conference on Computer Vision, pp. 1553–1560 (2013)
Turetken, E., Benmansour, F., Andres, B., Glowacki, P., Pfister, H., Fua, P.: Reconstructing curvilinear networks using path classifiers and integer programming. IEEE Trans. Pattern Anal. Mach. Intell. 38(12), 2515–2530 (2016)
Wegner, J.D., Montoya-Zegarra, J.A., Schindler, K.: A higher-order CRF model for road network extraction. In: Conference on Computer Vision and Pattern Recognition, pp. 1698–1705 (2013)
Wiedemann, C., Heipke, C., Mayer, H., Jamet, O.: Empirical evaluation of automatically extracted road axes. In: Empirical Evaluation Techniques in Computer Vision, pp. 172–187 (1998)
Wolterink, J., van Hamersvelt, R., Viergever, M., Leiner, T., Isgum, I.: Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier. Med.Image Anal. 51, 46–60 (2019)
Wu, D., et al.: A learning based deformable template matching method for automatic rib centerline extraction and labeling in CT images. In: Conference on Computer Vision and Pattern Recognition (2012)
Acknowledgements
DO received support from the Swiss National Science Foundation under Sinergia grant number 177237. MK was supported by the FWF Austrian Science Fund Lise Meitner grant no. M3374. The author’s version of the manuscript is available from arxiv.org.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Oner, D., Osman, H., Koziński, M., Fua, P. (2022). Enforcing Connectivity of 3D Linear Structures Using Their 2D Projections. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13435. Springer, Cham. https://doi.org/10.1007/978-3-031-16443-9_57
Download citation
DOI: https://doi.org/10.1007/978-3-031-16443-9_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16442-2
Online ISBN: 978-3-031-16443-9
eBook Packages: Computer ScienceComputer Science (R0)
