Abstract
Understanding the intensity characteristics of brain lesions is key for defining image-based biomarkers in neurological studies and for predicting disease burden and outcome. In this work, we present a novel foreground-based generative method for modelling the local lesion characteristics that can both generate synthetic lesions on healthy images and synthesize subject-specific pseudo-healthy images from pathological images. Furthermore, the proposed method can be used as a data augmentation module to generate synthetic images for training brain image segmentation networks. Experiments on multiple sclerosis (MS) brain images acquired on magnetic resonance imaging (MRI) demonstrate that the proposed method can generate highly realistic pseudo-healthy and pseudo-pathological brain images. Data augmentation using the synthetic images improves the brain image segmentation performance compared to traditional data augmentation methods as well as a recent lesion-aware data augmentation technique, CarveMix. The code will be released at https://github.com/dogabasaran/lesion-synthesis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bakshi, R., Ariyaratana, S., Benedict, R.H.B., Jacobs, L.: Fluid-attenuated inversion recovery magnetic resonance imaging detects cortical and juxtacortical multiple sclerosis lesions. Arch. Neurol. 58(5), 742–748 (2001). https://doi.org/10.1001/archneur.58.5.742
Bissoto, A., Perez, F., Valle, E., Avila, S.: Skin lesion synthesis with generative adversarial networks. In: Stoyanov, D., et al. (eds.) CARE/CLIP/OR 2.0/ISIC -2018. LNCS, vol. 11041, pp. 294–302. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01201-4_32
Bowles, C., Qin, C., Guerrero, R., et al.: Brain lesion segmentation through image synthesis and outlier detection. NeuroImage Clin. 16, 643–658 (2017). https://doi.org/10.1016/j.nicl.2017.09.003
Carass, A., Roy, S., Jog, A., et al.: Longitudinal multiple sclerosis lesion segmentation: resource and challenge. Neuroimage 148, 77–102 (2017). https://doi.org/10.1016/j.neuroimage.2016.12.064
Chartsias, A., Joyce, T., Papanastasiou, G., et al.: Disentangled representation learning in cardiac image analysis. Med. Image Anal. 58, 101535 (2019). https://doi.org/10.1016/j.media.2019.101535
Commowick, O., Istace, A., Kain, M., et al.: Objective evaluation of multiple sclerosis lesion segmentation using a data management and processing infrastructure. Sci. Rep. 8(1), 13650 (2018). https://doi.org/10.1038/s41598-018-31911-7
Dalton, C.M., Brex, P.A., Jenkins, R., et al.: Progressive ventricular enlargement in patients with clinically isolated syndromes is associated with the early development of multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 73(2), 141–147 (2002). https://doi.org/10.1136/jnnp.73.2.141
Dalton, C.M., Miszkiel, K.A., O’Connor, P.W., et al.: Ventricular enlargement in MS. Neurology 66(5), 693–698 (2006). https://doi.org/10.1212/01.wnl.0000201183.87175.9f
Ghasemi, N., Razavi, S., Nikzad, E.: Multiple sclerosis: pathogenesis, symptoms, diagnoses and cell-based therapy. Cell J. 19(191), 1–10 (2017). https://doi.org/10.22074/cellj.2016.4867
Guptha, S.H., Holroyd, E., Campbell, G.: Progressive lateral ventricular enlargement as a clue to Alzheimer’s disease. The Lancet 359(9322), 2040 (2002). https://doi.org/10.1016/S0140-6736(02)08806-2
Huang, H., Yu, P.S., Wang, C.: An introduction to image synthesis with generative adversarial nets (2018). https://doi.org/10.48550/ARXIV.1803.04469
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z
Jin, Q., Cui, H., Sun, C., Meng, Z., Su, R.: Free-form tumor synthesis in computed tomography images via richer generative adversarial network. Knowl.-Based Syst. 218, 106753 (2021). https://doi.org/10.1016/j.knosys.2021.106753
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396–4405 (2019). https://doi.org/10.1109/CVPR.2019.00453
Li, Q., Yu, Z., Wang, Y., Zheng, H.: TumorGAN: a multi-modal data augmentation framework for brain tumor segmentation. Sensors 20(15), 4203 (2020). https://doi.org/10.3390/s20154203
Luxenberg, J.S., Haxby, J.V., Creasey, H., Sundaram, M., Rapoport, S.I.: Rate of ventricular enlargement in dementia of the Alzheimer type correlates with rate of neuropsychological deterioration. Neurology 37(7), 1135 (1987). https://doi.org/10.1212/WNL.37.7.1135
Mao, X., Li, Q., Xie, H., et al.: Least squares generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821 (2017). https://doi.org/10.1109/ICCV.2017.304
Reed, S., Akata, Z., Yan, X., et al.: Generative adversarial text to image synthesis. In: Proceedings of the 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, New York, USA, 20–22 June 2016, vol. 48, pp. 1060–1069. PMLR, New York (2016)
Reinhold, J.C., Carass, A., Prince, J.L.: A structural causal model for MR images of multiple sclerosis. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 782–792. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_75
Salem, M., Valverde, S., Cabezas, M., et al.: Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-NET. IEEE Access 7, 25171–25184 (2019). https://doi.org/10.1109/ACCESS.2019.2900198
Styner, M., Lee, J., Chin, B., et al.: 3D segmentation in the clinic: a grand challenge II: MS lesion segmentation (2008). https://doi.org/10.54294/lmkqvm
Sun, L., Wang, J., Huang, Y., et al.: An adversarial learning approach to medical image synthesis for lesion detection. IEEE J. Biomed. Health Inform. 24(8), 2303–2314 (2020). https://doi.org/10.1109/JBHI.2020.2964016
Tang, H., Xu, D., Sebe, N., Yan, Y.: Attention-guided generative adversarial networks for unsupervised image-to-image translation. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019). https://doi.org/10.1109/IJCNN.2019.8851881
Xia, T., Chartsias, A., Tsaftaris, S.A.: Pseudo-healthy synthesis with pathology disentanglement and adversarial learning. Med. Image Anal. 64, 101719 (2020). https://doi.org/10.1016/j.media.2020.101719
Yun, S., Han, D., Chun, S., et al.: Cutmix: regularization strategy to train strong classifiers with localizable features. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6022–6031 (2019). https://doi.org/10.1109/ICCV.2019.00612
Zeng, C., Gu, L., Liu, Z., Zhao, S.: Review of deep learning approaches for the segmentation of multiple sclerosis lesions on brain MRI. Front. Neuroinform. 14, 610967 (2020). https://doi.org/10.3389/fninf.2020.610967
Zhang, X., et al.: CarveMix: a simple data augmentation method for brain lesion segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 196–205. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_19
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244
Acknowledgements
This work is supported by the UKRI CDT in AI for Healthcare http://ai4health.io (Grant No. EP/S023283/1). For the purpose of open access, the author has applied a ‘Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Basaran, B.D., Qiao, M., Matthews, P.M., Bai, W. (2022). Subject-Specific Lesion Generation and Pseudo-Healthy Synthesis for Multiple Sclerosis Brain Images. In: Zhao, C., Svoboda, D., Wolterink, J.M., Escobar, M. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2022. Lecture Notes in Computer Science, vol 13570. Springer, Cham. https://doi.org/10.1007/978-3-031-16980-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-16980-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16979-3
Online ISBN: 978-3-031-16980-9
eBook Packages: Computer ScienceComputer Science (R0)
