Abstract
Although federated learning has been widely used in collaborative training of machine learning models, its practical uses are still challenged by heterogeneous data across clients. To alleviate the impact of non-IID data issue, we present an adaptive clustered federated learning approach, \(\mathtt {AdaCFL}\), which can classify clients into suitable clusters according to their local data distribution and train a specialized model for the clients of each cluster. By exploiting the implicit connection between local model weights and data distribution on clients, \(\mathtt {AdaCFL}\) relies on partial selected model weights to measure the data similarity between clients and adaptively groups them into the optimal number of clusters. Experimental results on three benchmark datasets with various non-IID data settings demonstrate that \(\mathtt {AdaCFL}\) achieves comparably high model accuracy as the state-of-the-art works, yet with a significant reduction on the communication cost.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets i.e., MNIST [11], CIFAR-10 [10] and FashionMNIST [8], analysed during the current study are available from http://yann.lecun.com/exdb/mnist/, http://www.cs.toronto.edu/ ~ kriz/cifar.html and https://github.com/zalandoresearch/fashion-mnist
References
Bonawitz K, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, Kiddon C, Konečnỳ J, Mazzocchi S, McMahan B et al (2019) Towards federated learning at scale: System design. Proceedings of Machine Learning and Systems 1:374–388
Dasgupta S, Long PM (2005) Performance guarantees for hierarchical clustering. J Comput Syst Sci 70(4):555–569
Duan M, Liu D, Ji X, Liu R, Liang L, Chen X, Tan Y (2020) Fedgroup: Ternary cosine similarity-based clustered federated learning framework toward high accuracy in heterogeneous data. arXiv preprint arXiv:201006870
Gao H, Liu C, Yin Y, Xu Y, Li Y (2021a) A hybrid approach to trust node assessment and management for vanets cooperative data communication: Historical interaction perspective. IEEE Transactions on Intelligent Transportation Systems
Gao H, Yin Y, Han G, Zhao W (2021b) Edge computing: Enabling technologies, applications, and services. Transactions on Emerging Telecommunications Technologies 32(6)
Gao H, Zhou L, Kim JY, Li Y, Huang W (2021c) Applying probabilistic model checking to the behavior guidance and abnormality detection for mci patients under wireless sensor network. ACM Transactions on Sensor Networks https://doi.org/10.1145/3499426
Ghosh A, Chung J, Yin D, Ramchandran K (2020) An efficient framework for clustered federated learning. Adv Neural Inf Process Syst 33:19586–19597
Han X, Kashif R, Roland V (2017) The fashionmnist dataset. online: https://github.com/zalandoresearch/fashion-mnist
Hsieh K, Phanishayee A, Mutlu O, Gibbons P (2020) The non-iid data quagmire of decentralized machine learning. In: International Conference on Machine Learning, pp 4387–4398
Krizhevsky A, Hinton G, et al. (2014) The cifar-10 dataset. online: http://www.cstoronto.edu/~kriz/cifar.html
LeCun Y (1998) The mnist database of handwritten digits. online: http://yann lecun com/exdb/mnist/
Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2020a) Federated optimization in heterogeneous networks. vol 2, pp 429–450
Li X, Huang K, Yang W, Wang S, Zhang Z (2020b) On the convergence of fedavg on non-iid data. In: International Conference on Learning Representations
Liang T, Sheng X, Zhou L, Li Y, Gao H, Yin Y, Chen L (2021) Mobile app recommendation via heterogeneous graph neural network in edge computing. Appl Soft Comput 103:107162
Liu Y, Huang A, Luo Y, Huang H, Liu Y, Chen Y, Feng L, Chen T, Yu H, Yang Q (2020) Fedvision: An online visual object detection platform powered by federated learning. Proceedings of the AAAI Conference on Artificial Intelligence 34:13172–13179
Long M, Cao Y, Cao Z, Wang J, Jordan MI (2018) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell 41(12):3071–3085
Ma X, Xu H, Gao H, Bian M (2021) Real-time multiple-workflow scheduling in cloud environments. IEEE Trans Netw Serv Manage. https://doi.org/10.1109/TNSM.2021.3125395
McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics, pp 1273–1282
Mou L, Meng Z, Yan R, Li G, Xu Y, Zhang L, Jin Z (2016) How transferable are neural networks in nlp applications? In: Conference on Empirical Methods in Natural Language Processing, pp 479–489
Ouyang X, Xie Z, Zhou J, Huang J, Xing G (2021) Clusterfl: a similarity-aware federated learning system for human activity recognition. In: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, pp 54–66
Sattler F, Wiedemann S, Müller KR, Samek W (2019) Robust and communication-efficient federated learning from non-iid data. IEEE transactions on neural networks and learning systems 31(9):3400–3413
Sattler F, Müller KR, Samek W (2020) Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE transactions on neural networks and learning systems 32(8):3710–3722
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556
Smith V, Chiang CK, Sanjabi M, Talwalkar AS (2017) Federated multi-task learning. vol 30
Wang H, Kaplan Z, Niu D, Li B (2020) Optimizing federated learning on non-iid data with reinforcement learning. In: IEEE Conference on Computer Communications, pp 1698–1707
Wang K, Mathews R, Kiddon C, Eichner H, Beaufays F, Ramage D (2019) Federated evaluation of on-device personalization. arXiv preprint arXiv:191010252
Wang Z, Xu H, Liu J, Huang H, Qiao C, Zhao Y (2021) Resource-efficient federated learning with hierarchical aggregation in edge computing. In: IEEE Conference on Computer Communications, IEEE, pp 1–10
Xie M, Long G, Shen T, Zhou T, Wang X, Jiang J, Zhang C (2021) Multi-center federated learning. arXiv preprint arXiv:210808647
Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology 10(2):1–19
Yin Y, Cao Z, Xu Y, Gao H, Li R, Mai Z (2020a) Qos prediction for service recommendation with features learning in mobile edge computing environment. IEEE Transactions on Cognitive Communications and Networking 6(4):1136–1145
Yin Y, Huang Q, Gao H, Xu Y (2020b) Personalized apis recommendation with cognitive knowledge mining for industrial systems. IEEE Transactions on Industrial Informatics
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? vol 27
Zheng W, Yan L, Gou C, Wang FY (2021) Federated meta-learning for fraudulent credit card detection. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp 4654–4660
Zhu X, Wang J, Hong Z, Xiao J (2020) Empirical studies of institutional federated learning for natural language processing. In: Findings of the Association for Computational Linguistics: EMNLP, pp 625–634
Acknowledgements
This work was supported in part by International Cooperation Project of Shaanxi Province (No. 2020KW-004), the China Postdoctoral Science Foundation (No. 2017M613187), the Shaanxi Science and Technology Innovation Team Support Project under grant agreement (No. 2018TD-026), the China NSFC Grant (No.62172284) and the Natural Science Foundation of Guangdong (General Program No.2020A1515011502).
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Gong, B., Xing, T., Liu, Z. et al. Adaptive Clustered Federated Learning for Heterogeneous Data in Edge Computing. Mobile Netw Appl 27, 1520–1530 (2022). https://doi.org/10.1007/s11036-022-01978-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11036-022-01978-8