Abstract
Hospitals and research institutions may not be willing to share their collected medical data due to privacy concerns, transmission cost, and the intrinsic value of the data. Federated medical image analysis is thus explored to obtain a global model without access to the images distributed on isolated clients. However, in real-world applications, the local data from each client are likely non-i.i.d distributed because of the variations in geographic factors, patient demographics, data collection process, and so on. Such heterogeneity in data poses severe challenges to the performance of federated learning. In this paper, we introduce federated medical image analysis with virtual sample synthesis (FedVSS). Our method can improve the generalization ability by adversarially synthesizing virtual training samples with the local models and also learn to align the local models by synthesizing high-confidence samples with regard to the global model. All synthesized data will be further utilized in local model updating. We conduct comprehensive experiments on five medical image datasets retrieved from MedMNIST and Camelyon17, and the experimental results validate the effectiveness of our method. Our code is available at Link.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adnan, M., Kalra, S., Cresswell, J.C., Taylor, G.W., Tizhoosh, H.R.: Federated learning and differential privacy for medical image analysis. Sci. Rep. 12(1), 1–10 (2022)
Cetinkaya, A.E., Akin, M., Sagiroglu, S.: Improving performance of federated learning based medical image analysis in non-IID settings using image augmentation. In: 2021 International Conference on Information Security and Cryptology (ISCTURKEY), pp. 69–74. IEEE (2021)
He, C., et al.: FedML: a research library and benchmark for federated machine learning. arXiv preprint arXiv:2007.13518 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Jiang, M., Wang, Z., Dou, Q.: HarmoFL: harmonizing local and global drifts in federated learning on heterogeneous medical images. arXiv preprint arXiv:2112.10775 (2021)
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., Suresh, A.T.: SCAFFOLD: stochastic controlled averaging for federated learning. In: International Conference on Machine Learning, pp. 5132–5143. PMLR (2020)
Koh, P.W., et al.: WILDS: a benchmark of in-the-wild distribution shifts. In: International Conference on Machine Learning, pp. 5637–5664. PMLR (2021)
Li, D., Kar, A., Ravikumar, N., Frangi, A.F., Fidler, S.: Federated simulation for medical imaging. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 159–168. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_16
Li, D., Wang, J.: FedMD: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019)
Li, Q., He, B., Song, D.: Model-contrastive federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10713–10722 (2021)
Li, W., et al.: Privacy-preserving federated brain tumour segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 133–141. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32692-0_16
Lin, T., Kong, L., Stich, S.U., Jaggi, M.: Ensemble distillation for robust model fusion in federated learning. arXiv preprint arXiv:2006.07242 (2020)
Litjens, G., et al.: 1399 H &E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset. GigaScience 7(6), giy065 (2018)
Liu, Q., Chen, C., Qin, J., Dou, Q., Heng, P.A.: FedDG: federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1013–1023 (2021)
Liu, Q., Yang, H., Dou, Q., Heng, P.-A.: Federated semi-supervised medical image classification via inter-client relation matching. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 325–335. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_31
Liu, T., Siegel, E., Shen, D.: Deep learning and medical image analysis for covid-19 diagnosis and prediction. Ann. Rev. Biomed. Eng. 24 (2022)
Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based out-of-distribution detection. Adv. Neural. Inf. Process. Syst. 33, 21464–21475 (2020)
Luo, J., Wu, S.: FedSLD: federated learning with shared label distribution for medical image classification. arXiv preprint arXiv:2110.08378 (2021)
McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Miyato, T., Maeda, S.I., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1979–1993 (2018)
Qiu, Z., et al.: A deep learning approach for segmentation, classification, and visualization of 3-D high-frequency ultrasound images of mouse embryos. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 68(7), 2460–2471 (2021)
Reddi, S., et al.: Adaptive federated optimization. arXiv preprint arXiv:2003.00295 (2020)
Roth, H.R., et al.: Federated whole prostate segmentation in MRI with personalized neural architectures. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 357–366. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_34
Sahu, A.K., Li, T., Sanjabi, M., Zaheer, M., Talwalkar, A., Smith, V.: On the convergence of federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.061273 (2018)
Seo, H., Park, J., Oh, S., Bennis, M., Kim, S.L.: Federated knowledge distillation. arXiv preprint arXiv:2011.02367 (2020)
Taleb, A., Lippert, C., Klein, T., Nabi, M.: Multimodal self-supervised learning for medical image analysis. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) IPMI 2021. LNCS, vol. 12729, pp. 661–673. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78191-0_51
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., Khazaeni, Y.: Federated learning with matched averaging. arXiv preprint arXiv:2002.06440 (2020)
Wei, C., Shen, K., Chen, Y., Ma, T.: Theoretical analysis of self-training with deep networks on unlabeled data. arXiv preprint arXiv:2010.03622 (2020)
Wu, Y., Zeng, D., Wang, Z., Shi, Y., Hu, J.: Federated contrastive learning for volumetric medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 367–377. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_35
Xia, Y., et al.: Auto-FedAvg: learnable federated averaging for multi-institutional medical image segmentation. arXiv preprint arXiv:2104.10195 (2021)
Yang, D., et al.: Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan. Med. Image Anal. 70, 101992 (2021)
Yang, J., et al.: MedMNIST v2: a large-scale lightweight benchmark for 2D and 3D biomedical image classification. arXiv preprint arXiv:2110.14795 (2021)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 10(2), 1–19 (2019)
Zhu, W., Liao, H., Li, W., Li, W., Luo, J.: Alleviating the incompatibility between cross entropy loss and episode training for few-shot skin disease classification. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 330–339. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_32
Zhu, W., Luo, J., White, A.D.: Federated learning of molecular properties with graph neural networks in a heterogeneous setting. Patterns 100521 (2022)
Acknowledgement
This work was supported in part by NIH 1P50NS108676-01, NIH 1R21DE030251-01 and NSF award 2050842.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, W., Luo, J. (2022). Federated Medical Image Analysis with Virtual Sample Synthesis. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13433. Springer, Cham. https://doi.org/10.1007/978-3-031-16437-8_70
Download citation
DOI: https://doi.org/10.1007/978-3-031-16437-8_70
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16436-1
Online ISBN: 978-3-031-16437-8
eBook Packages: Computer ScienceComputer Science (R0)
