Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 3, No. 2, 2023 35 Hierarchical federated learning with mobile edge computing in the Internet of Vehicles Senwei Zhang, Fei Li *, Yi Zhang Cyberspace Security College, Chengdu University of Information Engineering, Chengdu, China * Corresponding author: Fei Li (Email: lifei@cuit.edu.cn) Abstract: Federated Learning is a distributed machine learning framework, which can be used in the Internet of Vehicles to train deep learning models without directly accessing the original data of mobile edge vehicle nodes. ECS can access massive data, but it has the characteristics of high latency and high communication overhead. However, mobile edge computing (MEC) platform can directly and efficiently communicate with mobile edge vehicle nodes. Combining the advantages of the two, a three-layer federated learning system of edge car network edge server cloud server is used. This system is supported by the HierFedProx algorithm and aggregates the model output of the edge car to the edge server to improve the model learning efficiency and reduce the global communication frequency. The experimental results show that the system can reduce the training time and improve the accuracy of the model compared with the federated learning without introducing the edge server. Keywords: Federated Learning; Federated Proximal; Mobile edge computing platform; V2X. 1. Introduction With the continuous development of the Internet of Vehicles and 5G technologies, as well as the continuous increase of the number of connected vehicles, a large number of edge vehicle nodes accessed in the network have generated large-scale operation data. The rise of machine learning provides a powerful tool for large-scale data processing and analysis, effectively improving the user experience. Although machine learning has a good experimental effect in the field of the Internet of Vehicles, it still faces the following problems: 1) From the perspective of privacy protection, there is a risk of information leakage in the transmission and storage of data in the Internet of Vehicles; 2) At the edge vehicle node, due to the limited communication and computing resources, it is difficult to ensure the efficiency of model training while quickly and effectively calculating and processing the data. In order to solve the problem of privacy protection of network computing scenarios for edge vehicles, federated learning came into being. Using this algorithm, a shared model is trained cooperatively without directly accessing the original data of the edge vehicle nodes. In the FedProx algorithm, the edge car node downloads a global machine learning model from the ECS, performs several local training, and then uploads the model to the ECS for global model aggregation. Repeat the process until the model reaches the expected accuracy. Although, federal learning will sink the training to the edge car node, so that the data will always remain on the edge car node, significantly improving the privacy of the edge car data. However, its iterative training process requires hundreds to thousands of rounds of communication to achieve the ideal model accuracy, which will lead to unbearable communication delay. There is a problem that the model is difficult to balance between communication efficiency and model performance. The mobile edge computing platform is introduced as the intermediate structure to reduce the communication rounds between the edge car node and the ECS. Mobile edge computing is characterized by high bandwidth and low latency, which greatly reduces the delay of network data transmission by being close to edge car nodes [1,2]. The federated learning algorithm based on mobile edge computing uses the nearest edge server as a parameter server, such as a base station (macro base station and small cell base station) [3], while the automobile nodes within the communication range of its server will cooperate to train the model, and the calculated delay is equivalent to the communication delay of the edge parameter server. Therefore, it is possible to seek a better balance between computing and communication. In literature [4], the author introduced mobile edge computing platform into federated migration learning, and obtained a personalized detection model without revealing data and privacy. In reference [5], the author proposed a federated optimization algorithm based on edge computing to solve the problem of large amount of computation when mobile devices implement the federated average algorithm. The local update process is divided into mobile edge car node and edge server to complete, and the global aggregation process is carried out between the edge server and ECS to reduce the total communication and computing costs of the edge car node. Combining the advantages of federated learning and mobile edge computing, we use a three-tier architecture of edge car network edge server cloud server federated learning, whose algorithm is HierFedProx. Compared with HierFAVG algorithm, HierFedProx algorithm has inconsistent distribution among edge data, which can improve the robustness and stability of model convergence. The three-tier architecture can use the edge server as the intermediary between the edge car network and the cloud server, which can significantly reduce the communication cost, while still using the rich data of the edge car network to improve the machine learning efficiency. 2. Research status 2.1. Mobile edge computing Mobile edge computing provides Internet service environment and cloud computing capability for 5G, WIFI and other wireless networks, and can meet the communication 36 requirements of ultra-high reliability and ultra-low delay of 5G function edge car nodes. In document [6], a reliable low delay vehicle communication architecture based on MEC (mobile edge computing platform) is proposed, which supports seamless handover of high mobility V2V communication, and proves that a flexible and active service MEC selection has better performance. For the research of MEC auxiliary communication, in literature [7], considering traffic load, computing and storage capacity, energy consumption and delay, it is proved that MEC can reduce resource leasing. In the paper [8], a new MEC based vehicle network framework is proposed, a task file transmission strategy with predictive V2V relay is designed, and an optimal predictive combined mode unloading scheme is proposed, which greatly reduces the unloading cost. In order to avoid interruptions in the unloading process, literature [9] proposed a dynamic unloading scheduling scheme based on MEC for the vehicle network, which divided the entire unloading task into small task units, and then calculated the number of units allocated and the optimal unloading ratio according to different constraints. With this scheduling scheme, the delay in each cell reaches a minimum value. The dynamic unloading scheduling scheme does not interrupt the unloading process. MEC has been widely used in the Internet of Vehicles to complete task unloading, data sharing and content caching of the Internet of Vehicles to meet the growing demand for computing intensive and delay sensitive automotive applications. 2.2. Federal Learning McMahan [10] first proposed federated learning technology in 2016. It is essentially a distributed machine learning framework, which can not only protect user privacy in the process of machine learning, but also realize data sharing in the case of raw data aggregation. According to the characteristics of the customer data set participating in the training, federal learning can be divided into horizontal federal learning and vertical federal learning [11,12]. The former mainly trains to extract the same feature descriptions of different objectives from multiple aspects; The latter mainly trains and extracts the same feature description of the same target from multiple aspects; The latter mainly trains federal migration learning to solve the problem of insufficient small label samples and data sets. As an emerging technology to meet this challenge, federated learning has broad application prospects in many fields. For example, edge computing and the Internet of Things, smart medicine, financial risk control, smart city, security sharing, etc. In the field of on-board system intrusion detection systems, [13] Yunlong Lu proposed a differential private asynchronous federated learning (DPAFL) scheme for the sharing of automotive network resources. In order to establish a secure and powerful federated learning scheme, the privacy of the updated local model is protected by incorporating local differential privacy into the gradient descent training scheme. [14] Subsequently, YunlongLu proposed a security scheme based on federated learning to reduce data leakage in VCP. In order to protect the privacy of learning process, a new random sub rumor update scheme is proposed. Based on professional federal learning, a two-phase solution is designed to reduce vehicle data leakage: intelligent data conversion and collaborative data leakage detection. This scheme can be deployed in multi-user and multi transmission channel application scenarios. 3. Internet of Vehicles framework based on mobile edge computing and federated learning 3.1. Framework overview In the Internet of Vehicles framework based on hierarchical federated learning, ECS combines different MEC platforms to obtain traffic data from the automotive edge network through 5G. MEC platform is represented by｛M1, M2, ..., MN｝, and data acquired by the edge automobile network is represented by ｛D1, D2, ..., DN｝. Based on the consideration of data privacy, we hope that ECS cannot directly aggregate all data D=D1∪D2∪...∪DN. The Internet of Vehicles framework of hierarchical federated learning aims to achieve fast and efficient data analysis and mining through federated learning and MEC platform under the condition of privacy protection. Its frame has three layers, as shown in Fig 1. The top layer is the ECS of 5G operators, which has a large amount of data computing and processing capabilities for training the server model. The bottom layer is the edge car node, which is widely distributed and communicates with each other. The MEC platform of the middle layer can obtain model parameters from the edge car nodes to aggregate models. MEC is not limited by resources and computing capacity, so the Internet of Vehicles framework based on hierarchical federated learning should be able to store and process data from all sensor networks, and should provide rapid response in a short time. According to this new system framework, using the HierFedProx algorithm, the key steps are as follows: τ After a local update, each edge server aggregates the training models of all its edge car nodes. Then at each τ After two edge aggregation models, ECS aggregates all edge server models. The data and information of any edge vehicle node will not be disclosed in the whole process. Figure 1. Hierarchical federal learning architecture of Internet of Vehicles 3.2. Edge car node - edge -cloud three-layer federated learning In the federated learning based on cloud edge automobile nodes, the ECS can aggregate the data of many clients, but the communication cost is large. The edge server can only aggregate a relatively small amount of data from edge car nodes, but the communication cost is low. Combining the advantages of both, we consider a hierarchical federated learning of edge car node edge cloud, supported by HierFedProx algorithm. This algorithm [15,16] can train the 37 model faster, achieve better communication calculation tradeoffs, and improve the accuracy of the model scene. The process of model learning is realized by minimizing a loss function, which is calculated according to the data on the edge car node. The purpose is to make the local update not too far away from the initial global model, and reduce the impact of no independent identically distributed (Non IID) on the premise of tolerating system heterogeneity. The loss function is used to evaluate the difference between the predicted value of the model and the real value. The better the loss function is, the better the model performance is generally. 2 ( ; ) ( ) 2 i ih f w     = + − (1) h ( ; ) ( )i if w      = + −（ ） (2) Among them, is the loss function of the federal average algorithm, is the local model update weight, is the global model weight issued by ECS, and is the near end penalty coefficient. When τ｜τ 1 ≠ 0; The evolution of local model parameters is as follows: h ( ; ) ( )i if w      = + −（ ） (3) 4. Experiment This section will introduce the experimental environment, experimental design, relevant models and results. Adjust the parameters of HierFedProx, and make an overall comparison between HierFedProx and FedProx. 4.1. Experimental environment and design The experimental computer is configured with 16GB RAM, i5-10400cpu@2.90GHz The processor, operating system is Window 10, and the experimental code is implemented using the Python framework. We have conceived a three-layer federated learning system of edge car network edge server cloud server. The system has a total of 50 edge car nodes, 5 MEC edge servers, and 1 ECS. And suppose that each MEC edge server has the same number of edge car nodes and the same training data. Based on the open source MNIST dataset, local training is conducted at each edge vehicle node. The batch size is 20, the initial learning rate is 0, and the exponential decay rate of each iteration is 0.995. The experimental results verify the effectiveness of this method. In the training process of HierFedProx, and will not only affect the training accuracy, but also directly affect the computing cost and communication efficiency of edge car nodes. Different sums are used in HierFedProx to evaluate performance. 4.2. Experimental results and analysis The experiment is nothing more than machine learning MNIST datasets in two non IID scenarios, namely edge independent ID and edge non IID. First, when the communication frequency between the mobile vehicle client and the ECS is fixed, that is, fixed time, the more frequent the communication between the mobile vehicle client and the edge server MEC is, that is, reduced, which can speed up the training process and improve the communication efficiency. As shown in Fig 2 and Fig 3, the overall observation of this figure shows that with the continuous increase of training rounds, the accuracy (Acc) under different parameter conditions shows an upward trend. In the Edge IID scene, no matter how the lines of four colors rise, the smaller the model accuracy is, the more accurate it is. No matter how it changes, as long as it is the same, the accuracy curve of the tested model is basically consistent, which means that increasing the communication frequency between MEC and ECS will not increase the training duration. As shown in Figure 3, when=60 in the Edge NIID scenario, increasing will delay the training process and affect the training accuracy of the model. The experimental results show that the communication frequency between the vehicle and the edge server can be further improved in order to reduce the high cost of traditional direct communication with the ECS. The training duration of minist dataset under the Edge ID and Edge NIID distribution is shown in Table 1. Figure 2. Test precision under MINIST data set(iid) Figure 3. Test precision under MINIST data set (Niid) Table 1. Raining Duration of Minist Dataset Under Edge ID and Edge NIID Distribution Edge-IID Edge-NIID totalT (s) totalT (s) 1 =6, 2 =10 4680 6120 1 =15, 2 =4 10140 11760 1 =30, 2 =2 13920 12360 1 =60, 2 =1 19680 28740 Next, we study another key quantity of the three-layer federated learning system, namely the training duration of the edge car node. In Table 1, we compared the cloud based federated learning of hierarchical federated learning under different scenarios when the data is fixed. According to the data in Table 1, we can clearly know that when the training duration reaches a certain accuracy, it decreases monotonously. This shows that from the perspective of reducing communication costs, the three-layer federated learning proposed by us is better than the two-layer federated learning based on cloud. 38 5. Summary This paper proposes a three-layer federated learning system of edge car network edge server cloud server, which is supported by the distributed algorithm HierFedProx. The system divides the local update into two parts: the edge car node and the edge car server. The model aggregation is completed on the edge server and the cloud server. Experiments show that compared with the traditional two- layer federated learning based on cloud, it can reduce the total communication and computing costs at the same time. In addition, it is also revealed that the selection of different key parameters in the HierFedProx algorithm can achieve different effects. References [1] Blanco B , Oscar Fajardo J , Giannoulakis I , et al. Technology pillars in the architecture of future 5G mobile networks: NFV, MEC and SDN[J]. Computer Standards and Interfaces . 2017(4):54. [2] Li Zuozhao, Liu Jinxu, LI, et al Application of mobile edge computing in Internet of Vehicles [J] Modern Telecommunications Technology, 2017, 47 (3): 5. [3] Kekki S, Featherstone W, Fang Y, et al. MEC in 5G networks. ETSI white paper, 2018, 28(2018): 1-28. [4] Fan Y , Li Y , Zhan M , et al. IoTDefender: A Federated Transfer Learning Intrusion Detection Framework for 5G IoT// 2020 IEEE 14th International Conference on Big Data Science and Engineering (BigDataSE). IEEE, 2020. [5] Ye Y , Li S , Liu F , et al. EdgeFed: Optimized Federated Learning Based on Edge Computing. IEEE Access, 2020, 8:209191-209198. [6] Zhou S , Netalkar P P , Chang Y , et al. The MEC-Based Architecture Design for Low-Latency and Fast Hand-Off Vehicular Networking// IEEE Vehicular Technology Conference (VTC). IEEE, 2018. [7] Xiao Z, Dai X, Jiang H, et al. Vehicular task offloading via heat-aware MEC cooperation using game-theoretic method. IEEE Internet of Things Journal, 2019, 7(3): 2038-2052. [8] Zhang Y, Lopez J, Wang Z. Mobile edge computing for vehicular networks [from the guest editors]. IEEE Vehicular Technology Magazine, 2019, 14(1): 27-108. [9] Wang H, Li X, Ji H, et al. Dynamic offloading scheduling scheme for MEC-enabled vehicular networks//2018 IEEE/CIC International Conference on Communications in China (ICCC Workshops). IEEE, 2018: 206-210. [10] Zhou Chuanxin, Sun Yi, Wang Degang, et al A Survey of Federated Learning Research Journal of Network and Information Security. [11] Yang Q , Liu Y , Chen T , et al. Federated Machine Learning: Concept and Applications. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2):1-19. [12] Yang Q , Liu Y , Chen T , et al. Federated Machine Learning: Concept and Applications. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2):1-19. [13] Lu Y, Huang X, Dai Y, et al. Differentially private asynchronous federated learning for mobile edge computing in urban informatics. IEEE Transactions on Industrial Informatics, 2019, 16(3): 2134-2143. [14] Lu Y, Huang X, Dai Y, et al. Federated learning for data privacy preservation in vehicular cyber-physical systems. IEEE Network, 2020, 34(3): 50-56. [15] Liu L, Zhang J, Song S H, et al. Client-edge-cloud hierarchical federated learning//ICC 2020-2020 IEEE International Conference on Communications (ICC). IEEE, 2020: 1-6. [16] Liu L, Zhang J, Song S H, et al. Edge-assisted hierarchical federated learning with non-iid data. arXiv preprint arXiv:1905.06641, 2019.