计算机科学 ›› 2022, Vol. 49 ›› Issue (3): 39-45.doi: 10.11896/jsjkx.210800054
赵罗成, 屈志昊, 谢在鹏
ZHAO Luo-cheng, QU Zhi-hao, XIE Zai-peng
摘要： 现有的联邦学习模型同步方法大多基于单层的参数服务器架构,难以适应当前异构无线网络场景,同时存在单点通信负载过重、系统延展性差等问题。针对这些问题,文中提出了一种面向边缘混合无线网络的联邦学习高效模型同步方法。在混合无线网络环境中,边缘移动终端将本地模型传输给附近的小型基站,小型基站收到边缘移动终端模型后执行聚合算法,并将聚合后的模型发送给宏基站完成全局模型更新。考虑到信道性能的异构性和数据传输在无线信道上的竞争关系,文中提出了一种新型的分组异步模型同步方法,并设计了基于传输速率感知的信道分配算法。在真实的数据集上进行了实验,实验结果表明,与传统的模型更新算法相比,所提基于分组异步模型更新的信道分配方法可以缩短25%~60%的训练通信时间,大幅度提升了联邦学习的训练效率。
中图分类号:
| [1]MOHANTA B K,JENA D,SATAPATHY U,et al.Survey onIoT security:Challenges and solution using machine learning,artificial intelligence and blockchain technology[J].Internet of Things,2020,11:100227-100244. [2]LI S,DA X L,ZHAO S.5G Internet of Things:A survey[J].Journal of Industrial Information Integration,2018,10:1-9. [3]QADRI Y A,NAUMAN A,ZIKRIA Y B,et al.The future ofhealthcare internet of things:a survey of emerging technologies[J].IEEE Communications Surveys & Tutorials,2020,22(2):1121-1167. [4]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial intelligence and statistics.PMLR,2017:1273-1282. [5]NIKNAM S,DHILLON H S,REED J H.Federated learning for wireless communications:Motivation,opportunities,and challenges[J].IEEE Communications Magazine,2020,58(6):46-51. [6]CHEN M,YANG Z,SAAD W,et al.A joint learning and communications framework for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2020,20(1):269-283. [7]KONČ ENÝ J,MCMAHAN H B,YU F X,et al.Federatedlearning:Strategies for improving communication efficiency[J].arXiv:1610.05492,2016. [8]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282. [9]CHEN M,POOR H V,SAAD W,et al.Convergence time optimization for federated learning over wireless networks[J].IEEE Transactions on Wireless Communications,2020,20(4):2457-2471. [10]TRAN N H,BAO W,ZOMAYA A,et al.Federated learningover wireless networks:Optimization model design and analysis[C]//IEEE INFOCOM 2019-IEEE Conference on Computer Communications.IEEE,2019:1387-1395. [11]PILLA L L.Optimal task assignment for heterogeneous federated learning devices[C]//2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS).IEEE,2021:661-670. [12]JIANG P,AGRAWAL G.A linear speedup analysis of distributed deep learning with sparse and quantized communication[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:2530-2541. [13]WADU M M,SAMARAKOON S,BENNIS M.Joint clientscheduling and resource allocation under channel uncertainty in federated learning[J].IEEE Transactions on Communications,2021,69(9):5962-5974. [14]YANG H H,LIU Z,QUEK T Q S,et al.Scheduling policies for federated learning in wireless networks[J].IEEE Transactions on Communications,2019,68(1):317-333. [15]XIA W,QUEK T Q S,GUO K,et al.Multi-armed bandit-based client scheduling for federated learning[J].IEEE Transactions on Wireless Communications,2020,19(11):7108-7123. [16]NISHIO T,YONETANI R.Client selection for federated lear-ning with heterogeneous resources in mobile edge[C]//2019 IEEE International Conference on Communications (ICC 2019).IEEE,2019:1-7. [17]QU Z,GUO S,WANG H,et al.Partial synchronization to acce-lerate federated learning over relay-assisted edge networks[J/OL].IEEE Transactions on Mobile Computing.https://ieeexplore.ieee.org/abstract/document/9439928. [18]ABAD M S H,OZFATURA E,GUNDUZ D,et al.Hierarchical federated learning across heterogeneous cellular networks[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).IEEE,2020:8866-8870. [19]LIU L,ZHANG J,SONG S H,et al.Client-edge-cloud hierarchical federated learning[C]//2020 IEEE International Confe-rence on Communications (ICC 2020).IEEE,2020:1-6. [20]YUAN J,XU M,MA X,et al.Hierarchical federated learning through lan-wan orchestration[J].arXiv:2010.11612,2020. [21]LIM W Y B,NG J S,XIONG Z,et al.Decentralized edge intelligence:a dynamic resource allocation framework for hierarchical federated learning[J].IEEE Transactions on Parallel and Distributed Systems,2021,33(3):536-550. [22]LUO S,CHEN X,WU Q,et al.Hfel:Joint edge association and resource allocation for cost-efficient hierarchical federated edge learning[J].IEEE Transactions on Wireless Communications,2020,19(10):6535-6548. [23]WU W,HE L,LIN W,et al.SAFA:A semi-asynchronous protocol for fast federated learning with low overhead[J].IEEE Transactions on Computers,2020,70(5):655-668. [24]XIE C,KOYEJO S,GUPTA I.Asynchronous federated optimization[J].arXiv:1903.03934,2019. [25]CHEN Y,NING Y,SLAWSKI M,et al.Asynchronous online federated learning for edge devices with non-iid data[C]//2020 IEEE International Conference on Big Data.IEEE,2020:15-24. [26]CHEN Y,SUN X,JIN Y.Communication-efficient federateddeep learning with layerwise asynchronous model update and temporally weighted aggregation[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(10):4229-4238. |
| [1] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于分层抽样优化的面向异构客户端的联邦学习 Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients 计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263 |
| [2] | 汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108 |
| [3] | 陈明鑫, 张钧波, 李天瑞. 联邦学习攻防研究综述 Survey on Attacks and Defenses in Federated Learning 计算机科学, 2022, 49(7): 310-323. https://doi.org/10.11896/jsjkx.211000079 |
| [4] | 闫萌, 林英, 聂志深, 曹一凡, 皮欢, 张兰. 一种提高联邦学习模型鲁棒性的训练方法 Training Method to Improve Robustness of Federated Learning 计算机科学, 2022, 49(6A): 496-501. https://doi.org/10.11896/jsjkx.210400298 |
| [5] | 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩. 基于DBSCAN聚类的集群联邦学习方法 Clustered Federated Learning Methods Based on DBSCAN Clustering 计算机科学, 2022, 49(6A): 232-237. https://doi.org/10.11896/jsjkx.211100059 |
| [6] | 杜辉, 李卓, 陈昕. 基于在线双边拍卖的分层联邦学习激励机制 Incentive Mechanism for Hierarchical Federated Learning Based on Online Double Auction 计算机科学, 2022, 49(3): 23-30. https://doi.org/10.11896/jsjkx.210800051 |
| [7] | 王鑫, 周泽宝, 余芸, 陈禹旭, 任昊文, 蒋一波, 孙凌云. 一种面向电能量数据的联邦学习可靠性激励机制 Reliable Incentive Mechanism for Federated Learning of Electric Metering Data 计算机科学, 2022, 49(3): 31-38. https://doi.org/10.11896/jsjkx.210700195 |
| [8] | 刘炜, 李东坤, 徐畅, 田钊, 佘维. 应急通信网络中基于粒子群优化的信道分配算法 Channel Assignment Algorithm Based on Particle Swarm Optimization in Emergency Communication Networks 计算机科学, 2021, 48(5): 277-282. https://doi.org/10.11896/jsjkx.200400042 |
| [9] | 吉晓祥, 沈航, 白光伟. 异构无线网络中基于非正交多址的可伸缩视频多播机制 Non-orthogonal Multiple Access Enabled Scalable Video Multicast in HetNets 计算机科学, 2021, 48(11): 356-362. https://doi.org/10.11896/jsjkx.200900080 |
| [10] | 陶洋,纪瑞娟,杨理,王进. 异构无线网络中动态优先级接纳控制算法研究 Study on Dynamic Priority Admission Control Algorithm in Heterogeneous Wireless Networks 计算机科学, 2020, 47(3): 242-247. https://doi.org/10.11896/jsjkx.190100089 |
| [11] | 郭斌, 余丹丹, 卢伟, 黄明和, 曾雅琳. 一种基于丢包预测的选择性网络编码策略 Selective Network Coding Strategy Based on Packet Loss Prediction 计算机科学, 2019, 46(11A): 399-404. |
| [12] | 刘萍,袁培燕. 移动自组织认知网络中的路由选择与信道分配方法 Routing Selection and Channel Assignment Method for Mobile Ad Hoc Cognitive Network 计算机科学, 2017, 44(3): 140-144. https://doi.org/10.11896/j.issn.1002-137X.2017.03.031 |
| [13] | 别玉霞,卜瑞杰,刘海燕. 多优先级的卫星网络信道分配算法 Channel Allocation Algorithm of Multi-priority Satellite Network 计算机科学, 2017, 44(3): 132-136. https://doi.org/10.11896/j.issn.1002-137X.2017.03.029 |
| [14] | 刘蔚,赵宇,陈锐. 无线Ad hoc网络中基于0-1优化的两步骤资源分配算法 Zero-One Integer Programming Based Optimization Model and Two-phase Resource Optimization Algorithm for Wireless Ad hoc Networks 计算机科学, 2017, 44(1): 103-108. https://doi.org/10.11896/j.issn.1002-137X.2017.01.020 |
| [15] | 李湘洋,赵杭生,赵小龙,张阳. 分布式多业务系统信道分配算法 Distributed Channel Allocation Algorithm for Multi-services Systems 计算机科学, 2016, 43(Z6): 272-275. https://doi.org/10.11896/j.issn.1002-137X.2016.6A.065 |
|
||