Abstract
Semi-supervised learning can be a promising approach in expediting the process of annotating medical images. In this paper, we use diffusion models to learn visual representations from multi-modal medical images in an unsupervised setting. These learned representations are then employed for the challenging downstream task of brain tumor segmentation. To avoid feature selection when using pixel-level classifiers, we propose fine-tuning the noise predictor network for semantic segmentation. We compare these methods against a supervised baseline over a varying number of training samples and evaluate their performance on a substantially larger test set. Our results show that, with less than 20 training samples, all methods outperform the supervised baseline across all tumor regions. Additionally, we present a practical use-case for patient-level tumor segmentation using limited supervision. The code we used and our trained diffusion model are publicly available (https://github.com/risc-mi/braintumor-ddpm).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amit, T., Shaharbany, T., Nachmani, E., Wolf, L.: Segdiff: Image segmentation with diffusion probabilistic models (2021)
Baid, U., et al.: The RSNA-ASNR-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification (2021)
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data 4, 170117 (2017)
Baranchuk, D., Voynov, A., Rubachev, I., Khrulkov, V., Babenko, A.: Label-efficient semantic segmentation with diffusion models. In: International Conference on Learning Representations (2022)
Cirillo, M.D., Abramian, D., Eklund, A.: Vox2Vox: 3D-GAN for brain tumour segmentation. In: Crimi, Alessandro, Bakas, Spyridon (eds.) BrainLes 2020. LNCS, vol. 12658, pp. 274–284. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72084-1_25
Dhariwal, P., Nichol, A.: Diffusion models beat GANS on image synthesis. In: Advances in Neural Information Processing Systems, vol. 34, pp. 8780–8794. Curran Associates, Inc. (2021)
Fiez, J.A., Damasio, H., Grabowski, T.J.: Lesion segmentation and manual warping to a reference brain: Intra- and interobserver reliability. Human Brain Mapping 9(4), 192–211 (2000)
Guo, X., Yang, Y., Ye, C., Lu, S., Xiang, Y., Ma, T.: Accelerating diffusion models via pre-segmentation diffusion sampling for medical image segmentation (2022)
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16000–16009 (2022)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems, vol. 33, pp. 6840–6851. Curran Associates, Inc. (2020)
Hoogeboom, E., Salimans, T.: Blurring diffusion models (2022)
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.: nnu-net: A self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021)
Kazerouni, A., et al.: Diffusion models for medical image analysis: A comprehensive survey (2022)
Li, D., Yang, J., Kreis, K., Torralba, A., Fidler, S.: Semantic segmentation with generative models: Semi-supervised learning and strong out-of-domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8300–8311 (2021)
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., Van Gool, L.: Repaint: Inpainting using denoising diffusion probabilistic models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11461–11471 (2022)
Luu, H.M., Park, S.H.: Extending nn-UNet for brain tumor segmentation. In: Brainlesion: Glioma. Multiple Sclerosis, Stroke and Traumatic Brain Injuries, pp. 173–186. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-09002-8_16
Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Nichol, A.Q., Dhariwal, P.: Improved denoising diffusion probabilistic models. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 8162–8171. PMLR (2021)
Nichol, A.Q., et al.: GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In: Proceedings of the 39th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 162, pp. 16784–16804. PMLR (2022)
Ouali, Y., Hudelot, C., Tami, M.: An overview of deep semi-supervised learning (2020)
Pinaya, W.H.L., et al.: Fast unsupervised brain anomaly detection and segmentation with diffusion models. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2022, pp. 705–714. Springer Nature Switzerland, Cham (2022). https://doi.org/10.1007/978-3-031-16452-1_67
Rissanen, S., Heinonen, M., Solin, A.: Generative modelling with inverse heat dissipation (2022)
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684–10695 (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D.J., Norouzi, M.: Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell. 45(4), 4713–4726 (2023)
Sharp, G.C., et al.: Vision 20/20: Perspectives on automated image segmentation for radiotherapy. Med Phys. 41(5), 050901 (2014)
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, pp. 2256–2265. PMLR, Lille (2015)
Wolleb, J., Bieder, F., Sandkühler, R., Cattin, P.C.: Diffusion models for medical anomaly detection. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2022, pp. 35–45. Springer Nature Switzerland, Cham (2022). https://doi.org/10.1007/978-3-031-16452-1_4
Wolleb, J., Sandkühler, R., Bieder, F., Valmaggia, P., Cattin, P.C.: Diffusion models for implicit image segmentation ensembles. In: Proceedings of the 5th International Conference on Medical Imaging with Deep Learning. Proceedings of Machine Learning Research, vol. 172, pp. 1336–1348. PMLR (2022)
Wu, J., et al.: Medsegdiff: Medical image segmentation with diffusion probabilistic model (2022)
Wyatt, J., Leach, A., Schmon, S.M., Willcocks, C.G.: Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 649–655 (2022)
Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 649–666. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_40
Zhang, Y., et al.: Datasetgan: Efficient labeled data factory with minimal human effort. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10145–10155 (2021)
Acknowledgments
This project is financed by research subsidies granted by the government of Upper Austria. RISC Software GmbH is Member of UAR (Upper Austrian Research) Innovation Network.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Alshenoudy, A., Sabrowsky-Hirsch, B., Thumfart, S., Giretzlehner, M., Kobler, E. (2023). Semi-supervised Brain Tumor Segmentation Using Diffusion Models. In: Maglogiannis, I., Iliadis, L., MacIntyre, J., Dominguez, M. (eds) Artificial Intelligence Applications and Innovations. AIAI 2023. IFIP Advances in Information and Communication Technology, vol 675. Springer, Cham. https://doi.org/10.1007/978-3-031-34111-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-031-34111-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34110-6
Online ISBN: 978-3-031-34111-3
eBook Packages: Computer ScienceComputer Science (R0)
