Abstract
Recent advances in bioimaging have provided scientists a superior high spatial-temporal resolution to observe dynamics of living cells as 3D volumetric videos. Unfortunately, the 3D biomedical video analysis is lagging, impeded by resource insensitive human curation using off-the-shelf 3D analytic tools. Herein, biologists often need to discard a considerable amount of rich 3D spatial information by compromising on 2D analysis via maximum intensity projection. Recently, pixel embedding based cell instance segmentation and tracking provided a neat and generalizable computing paradigm for understanding cellular dynamics. In this work, we propose a novel spatial-temporal voxel-embedding (VoxelEmbed) based learning method to perform simultaneous cell instance segmenting and tracking on 3D volumetric video sequences. Our contribution is in four-fold: (1) The proposed voxel embedding generalizes the pixel embedding with 3D context information; (2) Present a simple multi-stream learning approach that allows effective spatial-temporal embedding; (3) Accomplished an end-to-end framework for one-stage 3D cell instance segmentation and tracking without heavy parameter tuning; (4) The proposed 3D quantification is memory efficient via a single GPU with 12 GB memory. We evaluate our VoxelEmbed method on four 3D datasets (with different cell types) from the ISBI Cell Tracking Challenge. The proposed VoxelEmbed method achieved consistent superior overall performance (OP) on two densely annotated datasets. The performance is also competitive on two sparsely annotated cohorts with 20.6\(\%\) and 2\(\%\) of data-set having segmentation annotations. The results demonstrate that the VoxelEmbed method is a generalizable and memory-efficient solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Appel, K., Haken, W., et al.: Every planar map is four colorable. Bull. Am. Math. Soc. 82(5), 711–712 (1976)
Ballas, N., Yao, L., Pal, C., Courville, A.: Delving deeper into convolutional networks for learning video representations. arXiv preprint arXiv:1511.06432 (2015)
Cai, J., et al.: Accurate weakly-supervised deep lesion segmentation using large-scale clinical annotations: slice-propagated 3D mask generation from 2D RECIST. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 396–404. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_46
Cao, M., et al.: The single-cell transcriptional landscape of mammalian organogenesis. Nature 566(7745), 496–502 (2019)
von Chamier, L., Laine, R.F., Henriques, R.: Artificial intelligence for microscopy: what you should know. Biochem. Soc. Trans. 47(4), 1029–1040 (2019)
Chen, B.C., et al.: Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346(6208), 1257998 (2014)
Condeelis, J., Pollard, J.W.: Macrophages: obligate partners for tumor cell migration, invasion, and metastasis. Cell 124(2), 263–266 (2006)
Debeir, O., Van Ham, P., Kiss, R., Decaestecker, C.: Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes. IEEE Trans. Med. imaging 24(6), 697–711 (2005)
Meijering, E., Dzyubachyk, O., Smal, I.: Methods for cell and particle tracking. Methods Enzym. 504, 183–200 (2012)
ISBI: Isbi cell tracking challenge benchmark leader boarder (2021). http://celltrackingchallenge.net/latest-ctb-results/
Jiang, C., Tsai, Y.J.: Enhanced crack segmentation algorithm using 3D pavement data. J. Comput. Civil Eng. 30(3), 04015050 (2016)
Jiang, R., Gouvea, J., Hammer, D., Aeron, S.: Automatic coding of students’ writing via contrastive representation learning in the wasserstein space. arXiv preprint arXiv:2011.13384 (2020)
Jin, B., Cruz, L., Goncalves, N.: Deep facial diagnosis: deep transfer learning from face recognition to facial diagnosis. IEEE Access 8, 123649–123661 (2020)
Li, M., Chen, X., Li, X., Ma, B., Vitányi, P.M.: The similarity metric. IEEE Trans. Inform. Theory 50(12), 3250–3264 (2004)
Liu, Q., et al.: Towards annotation-free instance segmentation and tracking with adversarial simulations. arXiv preprint arXiv:2101.00567 (2021)
Liu, T.L., et al.: Observing the cell in its native state: imaging subcellular dynamics in multicellular organisms. Science 360(6386), eaaq1392 (2018)
Magnusson, K.E.: Segmentation and tracking of cells and particles in time-lapse microscopy. Ph.D. thesis, KTH Royal Institute of Technology (2016)
Matula, P., Maška, M., Sorokin, D.V., Matula, P., Ortiz-de Solórzano, C., Kozubek, M.: Cell tracking accuracy measurement based on comparison of acyclic oriented graphs. PloS ONE 10(12), e0144959 (2015)
Meijering, E.: A bird’s-eye view of deep learning in bioimage analysis. Comput. Struct. Biotech. J. 18, 2312 (2020)
Microsoft: Azure NC-series (2020). https://docs.microsoft.com/en-us/azure/virtual-machines/nc-series
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation, pp. 483–499 (2016)
Niwattanakul, S., Singthongchai, J., Naenudorn, E., Wanapu, S.: Using of Jaccard coefficient for keywords similarity. In: Proceedings of the International Multiconference of Engineers and Computer Scientists, vol. 1, pp. 380–384 (2013)
NVIDIA: Nvidia, V. (2013). tesla k20 gpu accelerator board specification (2015). https://www.nvidia.com/content/PDF/kepler/tesla-k20-active-bd-06499-001-v03.pdf
Ong, E.Z., et al.: A dynamic immune response shapes Covid-19 progression. Cell Host Microbe 27(6), 879–882 (2020)
Payer, C., Štern, D., Feiner, M., Bischof, H., Urschler, M.: Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks. Med. Image Anal. 57, 106–119 (2019)
Ulman, V., et al.: An objective comparison of cell-tracking algorithms. Nat. Methods 14(12), 1141–1152 (2017)
Wan, Y., McDole, K., Keller, P.J.: Light-sheet microscopy and its potential for understanding developmental processes. Annu. Rev. Cell Dev. Biol. 35, 655–681 (2019)
Yuan, W., Xu, W.: Neighborloss: a loss function considering spatial correlation for semantic segmentation of remote sensing image. IEEE Access 9, 75641–75649 (2021)
Zhao, M., Chang, C.H., Xie, W., Xie, Z., Hu, J.: Cloud shape classification system based on multi-channel CNN and improved FDM. IEEE Access 8, 44111–44124 (2020)
Zhao, M., et al.: Faster mean-shift: GPU-accelerated clustering for cosine embedding-based cell segmentation and tracking. Med. Image Anal. 71, 102048 (2021)
Zhou, X., Wong, S.T.: High content cellular imaging for drug development. IEEE Signal Process. Mag. 23(2), 170–174 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, M. et al. (2021). VoxelEmbed: 3D Instance Segmentation and Tracking with Voxel Embedding based Deep Learning. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds) Machine Learning in Medical Imaging. MLMI 2021. Lecture Notes in Computer Science(), vol 12966. Springer, Cham. https://doi.org/10.1007/978-3-030-87589-3_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-87589-3_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87588-6
Online ISBN: 978-3-030-87589-3
eBook Packages: Computer ScienceComputer Science (R0)
