Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 9, No. 2, 2024 193 Autonomous Driving System Driven by Artificial Intelligence Perception Fusion Yong Wang1, *, Shuqian Du2, Qi Xin3, Yuhang He4, Wenpin Qian5 1Information Technology, University of Aberdeen, Aberdeen, United Kingdom 2Information Studies, Trine University, Phoenix, AZ, USA 3Management Information Systems, University of Pittsburgh, Pittsburgh, PA, USA 4Computer Science and Technology, Tianjin University of Technology, Tianjin, China 5Information Science, Trine University, Phoenix AZ, USA *Corresponding author: fredia4jane@gmail.com Abstract: Perception, as the information input module of the automatic driving system, determines the lower limit of the entire automatic driving system. Both autonomous driving perception and robot perception are constantly approaching the real physical world through digital methods, and this real physical world is only limited to the scope of human perception, such as lane lines, traffic lights, driving obstacles, and so on. The main premise of this process is that humans already know the categories or properties of the physical world, and only allow machines and systems to replicate human responses. Whether it is a pure visual route or a multi-source fusion route, the essence is the difference between the perceptual system schemes, one focusing on the vertical and the other on the horizontal. The pure vision solution represented by Tesla or the multi-source sensor fusion file represented by Waymo. In fact, the perception module of the automatic driving system usually has multiple sensors to achieve information redundancy and information complementarity through multiple dimensions, but there is the possibility of information conflict between different sensors. This paper aims at the advantages of perception-driven artificial intelligence to achieve breakthroughs in autonomous driving innovation, and analyzes how perception fusion drive is applied to the practical application of autonomous driving, so as to analyze the future development prospects of artificial intelligence. Keywords: Perception fusion; Autonomous driving innovation; Artificial intelligence (AI); Perception; Information redundancy. 1. Introduction Artificial Intelligence Generated Content (AIGC) refers to a technology that generates relevant content with appropriate generalization ability through the learning and recognition of existing data based on artificial intelligence techniques such as generating adversarial networks and large pre-trained models. The core idea of AIGC technology is to use artificial intelligence algorithms to generate content with certain creativity and quality. By training the model and learning from large amounts of data, AIGC can generate content related to the input conditions or instructions. The development of AIGC can be traced back to 1950 when Alan Turing proposed the famous "Turing test" in his paper Computing Machinery and Intelligence. A test method is given to determine whether a machine has "intelligence", that is, whether the machine can imitate the way of human thinking to "generate" content. With the accumulation of valuable and effective data, the substantial improvement of computing power, and the proposal and application of deep learning algorithms, today's artificial intelligence technology has been popularized in many industries, and plays an important role in the corresponding scenarios[1]. For example, Natural Language Processing (NLP), computer vision, recommendation system, predictive analysis, etc., subvert the production and life mode in the corresponding scene, and play an important role in bringing great convenience to human society. The principle of AIGC technology is introduced, and the two generation models of single-mode and multi-mode are introduced. The definition, advantages, application in edge cloud computing, Web3.0, meta-universe, challenges, security, privacy, intellectual property threats and other aspects of AIGC are discussed in detail, and the future direction is given[2]. Taking ChatGPT as an example, the opportunities and challenges of artificial intelligence and super-large pre-training models are presented. Today, autonomous driving technology has become the future direction of the automotive industry. The application of autonomous driving technology can comprehensively improve the safety and comfort of car driving, meet higher level market demand, and promote the upgrading of industrial science and technology. Thanks to the development of artificial intelligence technology, autonomous driving technology has achieved breakthrough improvements in environmental perception, accurate positioning, decision- making and planning, control and execution. Among them, based on the research and development of artificial intelligence applications, the key technical competition points include high-precision maps, sensor fusion, voice and image recognition. Foreign autonomous vehicle research and development process is earlier, has gradually achieved commercial application, international top companies in the field of artificial intelligence autonomous driving research in the world's leading position[3]. And China's autonomous vehicle research and development is also gradually rising. Government encouragement policies and financial support have become an important driving force for the development of autonomous vehicles in China. This paper gives a more comprehensive introduction from the definition of autonomous vehicles, research and development process to the technical application of artificial intelligence in 194 autonomous vehicles, and the global autonomous driving market, and summarizes the research status of key domestic scientists, in order to understand the current development status of China's industry and international competition positioning. 2. Related Work Multimodal fusion is an important task in sensing autonomous driving system. In this paper, a multi-modal sensing method for automatic driving will be described in detail. Including LiDAR and camera to solve object detection and semantic segmentation tasks. 2.1. Multimodal perceptual fusion Multimodal fusion is an important task in sensing autonomous driving system. In this paper, a multi-modal sensing method for automatic driving will be described in detail. Including LiDAR and camera to solve object detection and semantic segmentation tasks. Why do we need multi- modal fusion? In a complex driving environment, a single sensor information is not enough to effectively process changes in the scene. For example, in the case of extremely bad weather (heavy rain, sandstorm) with low visibility, it is completely impossible to rely on the RGB images fed back by the camera to give feedback on changes in the environment. In the ordinary road environment, such as traffic lights, color cones, etc., only relying on Lidar information can not be effectively identified, but also need to combine the RGB information brought by the camera to effectively process[4- 6]. Therefore, in the task of automatic driving perception scene, the complementarity of different modal information will be more important. Multimodal fusion can be used in many scenarios, such as 2D/3D object detection, semantic segmentation, and Tracking tasks. In these tasks, it is the work of information interaction and fusion between modes. The acquisition of information from sensors is becoming more and more efficient and accurate, the cost is being compressed lower and lower, and the multi-modal fusion method in the perception task in autonomous driving has been a rapid development opportunity[7]. In general, some tasks can be viewed as driving perceptual tasks, including object detection, semantic segmentation, deep completion, and prediction. Figure 1. Multiple integrated drive task models Target detection: For self-driving cars, understanding their surroundings is crucial. Driverless cars need to detect stationary and moving obstacles on the road to ensure safe driving. Object detection is a traditional computer vision task that is widely used in autonomous driving systems [61108]. The researchers built such a framework for obstacle detection (cars, pedestrians, cyclists, etc.), traffic light detection, traffic sign detection, etc. In general, object detection uses rectangles or cuboids represented by parameters to tightly bind instances of predefined categories, such as cars or pedestrians, which needs to be excellent at both positioning and classification[8- 10]. Due to the lack of deep channels, 2D object detection is often simply expressed as (x; y; h; w; c), while the 3D object detection bounding box is usually represented as (x; y; z; h; w; l;; c). Semantic segmentation: In addition to object detection, many autonomous driving perception tasks can be formulated as semantic segmentation. For example, free-space detection is a fundamental module in many autonomous driving systems that divide ground pixels into actuable and non-actuable parts. Some lane detection methods also use multi-class semantic segmentation masks to represent different lanes on a road. The essence of semantic segmentation is to cluster the basic components of input data, such as pixels and 3D points, into multiple regions containing specific semantic information. Specifically, semantic segmentation refers to a given set of data, such as image pixel DI = fd1; d2; : : :; dng or LiDAR 3D point cloud DL = fd1; d2; : : :; dng and a predetermined set of candidate labels Y = fy1; y2; y3; : : :; ykg, we use a model to assign each pixel or point di the probability of selecting one or all of k semantic labels[10-11]. 2.2. Self-driven fusion mode Data-level fusion or early fusion methods directly fuse raw sensor data of different modes through spatial alignment. Feature level fusion or deep fusion methods mix cross-modal data in feature space by concatenation or element by element multiplication. The goal-level fusion method combines the model's predictions in each mode to make the final decision. 195 Figure 2. Fusion model methodology Early Fusion is generally the fusion of LIDAR data and Image images or the fusion of LIDAR data and Image features in two ways. The following figure shows the interaction process between this branch of LiDAR and the early information of Image information. This method can be used in reflectance, voxelized tensor, front-view/ range-view/ BEV, and pseudo-point clouds. Although the features of Image are different at each stage, they are highly related to the information of LiDAR[13-14]. Therefore, LiDAR information +Image feature fusion can also be effectively fused. Figure 3. Early Fusion Fusion model From an image point of view, the definition of a data-level picture in a strict sense can only contain RGB or Gray data. In fact, this definition is lack of universality and rationality, but also relatively limited. So our pattern should be larger, and the data level can not only be images, but also feature maps. Compared with the traditional early fusion definition, the definition of camera data is not limited to image, but also includes the feature information. The feature information is selected and fused consciously to obtain a semantically more closely connected input data, and then the data set is put into the network for feature extraction[15]. Whether the data type is converted into a consistent one directly and then concat into a whole, or the feature information of LiDAR and Image is fused, or the two are semantically connected before the feature becomes an input, these are all operations of Early Fusion. In fact, the advantages of such an input integration operation are naturally simple and easy to deploy. 2.3. 2.3 Multi-sensor fusion problem modeling In the fusion perception process of the autonomous driving self-driven sensor, the following formula can be used to describe the mapping and positioning: (1) Among them: Sensort for t moment sensor system to collect information, usually contains GNSST, imagest pointloudt,... ; ·R is the rotation matrix of the current self-propelled coordinate system relative to the world coordinate system, and t is the corresponding translation vector; ·landmarks are characteristic elements of road environment at time t, expressed in the form of semantics or feature points; (2) Among them: Sensort for t moment sensor system to collect information, usually contains {GNSST, imagest, pointclouds.. -}; objectst is a target-level obstacle in the environment at time t. Attributes such as position, speed, acceleration, and category can be used to describe an independent entity. scenariot is a semantic-level element description in the T- moment environment, which cannot be expressed as an independent obstacle form, such as a construction area or rainwater scene: For the perception problem, there are two main ways to achieve it: pre-fusion and post-fusion. To sum up, the combination of artificial intelligence perception fusion drive in the field of automatic driving has brought many advantages and roles[16]. First, it allows the autonomous driving system to collect environmental information using multiple sensors such as cameras, lidar and radar, thereby improving the system's perception of the surrounding environment, which in turn enhances driving safety and efficiency. Secondly, by processing and integrating these perceptual data with artificial intelligence technology, the autonomous driving system can more accurately identify and understand various elements on the road, such as vehicles, pedestrians, traffic signals, etc., so as to make more reliable decisions and plan driving paths. In addition, perceptual fusion drive also helps reduce the risk of misjudgment and misoperation of the system in complex environments, improving the reliability and stability of autonomous driving technology[17]. In summary, artificial intelligence perception fusion drive provides important support for the development of the field of autonomous driving, and promotes the continuous innovation and progress of autonomous driving technology. 3. Model and Methodology 3.1. Model description low-rank matrix factorization (LMF) : Is a tensor fusion 196 method, by using low-rank decomposition factors to carry out tensor fusion, avoiding the high dimensional problem caused by direct tensor complementary fusion. For more on LMF, see Low-rank Multimodal Fusion with Modality-Specific Factors. Figure 4. low-rank matrix factorization (LMF) Input tensor: (3) A linear layer g(·) produces a vector representation: (4) Where W is the weight and b is the offset: (5) If h is obtained by calculating Z, the dimension of m modes 2 is dx dzx... xdm, this low-rank decomposition instead of the original vector complement method, can directly obtain h, without computing the name of the higher dimension, so that it can be easily extended to the case of a large number of modes. In fact, in the semantic segmentation model, the deeper the network, the larger the receptive field, but the more information is lost in the decoder. Therefore, the retrieval of edge features is extremely important. U-Net uses the method of jump connection to fuse the level feature map in the encoder with the recovery results of the deconvolution layer to realize the edge feature retrieval. With reference to this U- shaped structure, we connect multimodal homologous features to skip connections while using them for deconvolution layer calculations. The disadvantage is that the concatenated feature tensor can be very thick. Taking KITTI data set as an example, the size of the original image is 1392×512, the size of the feature map after four downsamples is 87×32, and the dimension of the feature map of the fourth layer of ResNet-50 is 1024[18-22]. 3.2. Optimize the network learning algorithm for model training Efficient free space detection algorithm is very important for the deployment of intelligent driving system. However, there is little work discussing the pruning of semantic segmentation neural networks for free-space detection tasks. Pruning methods designed for classification tasks have been directly applied to segmentation neural networks, and filters are pruned in the backbone network on ImageNet and transferred to the segmentation network. This paper introduces a multi-task channel pruning method to obtain lightweight semantic segmentation networks. Assume that Ci is the I-th convolution layer of the pre- trained CNN model[23]. The weights in Ci can be expressed as WCi = {w1, w2,... , wni} ε Rni×ni−1×ki×ki, where ni represents the number Ci and the number of filters in ki represents the kernel size. The input feature map is represented as Xi ={x1, x2,... , xni}εRb×ni−1×hi×wi, where b is the batch size and hi and wi are the height and width of the feature map. Filter pruning is designed to identify and remove less important weights from WCi Settings, which can be formulated as optimization problems[20-21]: (6) Where F() measures the importance of the weights in the CNN. Delta is a filter, 1 if wi,j is important; If wi,j is not important, it is 0. Minimizing p is removing the least important weight in Ci. For autonomous vehicles, free-space detection is an important part of visual perception. In recent years, with the development of multi-modal convolutional neural networks (CNNS), the performance of semantic segmentation algorithms for driving scenes has been significantly improved. Therefore, most free-space detection algorithms are developed based on multiple sensors. The algorithm introduced in this paper first introduces a lightweight multimodal free space detection network with fewer convolution operators and smaller feature graphs[24- 28]. The parameters of the model are then reduced by filter 197 pruning and 8-bit quantization. Finally, the model is transplanted to the vehicle domain controller to make independent prediction in low-power devices. 4. Conclusion The combination of artificial intelligence perception fusion drive in the field of autonomous driving has brought huge advantages and roles to autonomous driving technology. Through the data fusion of multiple sensors and the processing of artificial intelligence, the autonomous driving system can more accurately perceive the surrounding environment, thus improving the safety and efficiency of driving. Perceptual fusion drive also helps reduce the risk of misjudgment and misoperation in complex environments, further improving the reliability and stability of autonomous driving technology. In summary, artificial intelligence perception fusion drive provides important support for the development of autonomous driving technology, and promotes the continuous innovation and progress of autonomous driving technology. Multimodal sensing fusion is an important task in automatic driving system. The use of multiple sensors for data fusion, such as LiDAR and cameras, can effectively deal with complex scene changes and improve the environment perception ability of autonomous driving systems. In autonomous driving perceptual tasks, the complementarity of different modal information becomes more important. Through data level fusion, feature level fusion and target level fusion, the information of different sensors can be better integrated. Therefore, the application prospect of multi-modal perception fusion method in the field of automatic driving is broad, and it will provide important support for the development of automatic driving technology in the future. Acknowledgment Here, I would like to thank Mr. Cankun Wang for his contribution and inspiration to this article. Mr. Cankun Wang in his article "Based on Intelligent" published in the International Journal of Computer Science and Information Technology Advertising Recommendation and Abnormal Advertising Monitoring System in the Field of Machine Learning "(Published in December 2023, Volume 1, 1, page 17-23, https://doi.org/10.62051/ijcsit.v1n1.03), and explore the field of machine learning ads recommendation and abnormal intelligent monitoring system of related problems. His research not only provides valuable reference and inspiration for this paper, but also provides important support and guidance for the deep learning and artificial intelligence related content of this paper. At the same time, I would like to especially thank Mr. Cankun Wang for his opinions and data inspiration in the field of deep learning and artificial intelligence, which have played an important role in promoting the research work of this paper. Finally, I would like to thank him for his review and suggestions, which have played an important role in the improvement and improvement of this paper. Once again, I would like to express my sincere thanks to Mr. Cankun Wang! References [1] Tan, Kai, et al. “Integrating Advanced Computer Vision and AI Algorithms for Autonomous Driving Systems”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Jan. 2024, pp. 41-48, doi:10.53469/jtpes.2024.04(01).06. [2] Chen, Wangmei, et al. “Applying Machine Learning Algorithm to Optimize Personalized Education Recommendation System”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Feb. 2024, pp. 101-8, doi:10.53469/jtpes.2024.04(01).14. [3] “Implementation of Computer Vision Technology Based on Artificial Intelligence for Medical Image Analysis”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 69-76, https://doi.org/10.62051/ijcsit.v1n1.10. [4] Dong, Xinqi, et al. “The Prediction Trend of Enterprise Financial Risk Based on Machine Learning ARIMA Model”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Jan. 2024, pp. 65-71, doi:10.53469/jtpes.2024.04(01).09. [5] “A Deep Learning-Based Algorithm for Crop Disease Identification Positioning Using Computer Vision”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 85-92, https://doi.org/10.62051/ijcsit.v1n1.12. [6] Du, S., Li, L., Wang, Y., Liu, Y., & Pan, Y. (2023). Application of HPV-16 in Liquid-Based thin Layer Cytology of Host Genetic Lesions Based on AI Diagnostic Technology Presentation of Liquid. Journal of Theory and Practice of Engineering Science, 3(12), 1-6. [7] Xin, Q., He, Y., Pan, Y., Wang, Y., & Du, S. (2023). The implementation of an AI-driven advertising push system based on a NLP algorithm. International Journal of Computer Science and Information Technology, 1(1), 30-37. [8] He, Yuhang, et al. “Intelligent Fault Analysis With AIOps Technology”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Feb. 2024, pp. 94-100, doi:10.53469/jtpes.2024.04(01).13. [9] Chen , J., Xiong, J., Wang, Y., Xin, Q., & Zhou, H. (2024). Implementation of an AI-based MRD Evaluation and Prediction Model for Multiple Myeloma. Frontiers in Computing and Intelligent Systems, 6(3), 127- 131. https://doi.org/10.54097/zJ4MnbWW [10] Tan, Kai, et al. “Integrating Advanced Computer Vision and AI Algorithms for Autonomous Driving Systems”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Jan. 2024, pp. 41-48, doi:10.53469/jtpes.2024.04(01).06. [11] “Exploring New Frontiers of Deep Learning in Legal Practice: A Case Study of Large Language Models”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 131-8, https://doi.org/10.62051/ijcsit.v1n1.18. [12] Development of Machine Learning and Artificial Intelligence in Toxic Pathology. (2024). Frontiers in Computing and Intelligent Systems, 6(3), 137- 141. https://doi.org/10.54097/Be1ExjZa [13] Jili Qian, et al. “Analysis and Diagnosis of Hemolytic Specimens by AU5800 Biochemical Analyzer Combined With AI Technology”. Frontiers in Computing and Intelligent Systems, vol. 6, no. 3, Jan. 2024, pp. 100-3, https://doi.org/10.54097/qoseeQ5N. [14] “A Deep Learning-Based Algorithm for Crop Disease Identification Positioning Using Computer Vision”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 85-92, https://doi.org/10.62051/ijcsit.v1n1.12. [15] K. Jin, Z. Z. Zhong and E. Y. Zhao, "Sustainable Digital Marketing Under Big Data: An AI Random Forest Model Approach," in IEEE Transactions on Engineering Management, vol. 71, pp. 3566-3579, 2024, doi: 10.1109/TEM.2023.3348991. 198 [16] Jin, Keyan. "Impacts of Word of Mouth (WOM) on E-Business Online Pricing." JGIM vol.31, no.3 2023: pp.1-17. http://doi.org/10.4018/JGIM.324813 [17] Pan, Yiming, et al. “Application of Three-Dimensional Coding Network in Screening and Diagnosis of Cervical Precancerous Lesions”. Frontiers in Computing and Intelligent Systems, vol. 6, no. 3, Jan. 2024, pp. 61-64, https://doi.org/10.54097/mi3VM0yB. [18] Wei, Kuo, et al. “Strategic Application of AI Intelligent Algorithm in Network Threat Detection and Defense”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Jan. 2024, pp. 49-57, doi:10.53469/jtpes.2024.04(01).07. [19] “Unveiling the Future Navigating Next-Generation AI Frontiers and Innovations in Application”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 147-56, https://doi.org/10.62051/ijcsit.v1n1.20. [20] Zong, Yanqi, et al. “Improvements and Challenges in StarCraft II Macro-Management A Study on the MSC Dataset”. Journal of Theory and Practice of Engineering Science, vol. 3, no. 12, Dec. 2023, pp. 29-35, doi:10.53469/jtpes.2023.03(12).05. [21] An Overview of the Development of Stereotactic Body Radiation Therapy. (2024). Frontiers in Computing and Intelligent Systems, 6(3), 56- 60. https://doi.org/10.54097/09nIy12x. [22] Zheng, Jiajian, et al. “The Credit Card Anti-Fraud Detection Model in the Context of Dynamic Integration Selection Algorithm”. Frontiers in Computing and Intelligent Systems, vol. 6, no. 3, Jan. 2024, pp. 119-22, https://doi.org/10.54097/a5jafgdv. [23] Du, Shuqian, et al. “Application of HPV-16 in Liquid-Based Thin Layer Cytology of Host Genetic Lesions Based on AI Diagnostic Technology Presentation of Liquid”. Journal of Theory and Practice of Engineering Science, vol. 3, no. 12, Dec. 2023, pp. 1-6, doi:10.53469/jtpes.2023.03(12).01. [24] Pan, Yiming, et al. “Application of Three-Dimensional Coding Network in Screening and Diagnosis of Cervical Precancerous Lesions”. Frontiers in Computing and Intelligent Systems, vol. 6, no. 3, Jan. 2024, pp. 61-64, https://doi.org/10.54097/mi3VM0yB. [25] Wang, Sihao, et al. “Diabetes Risk Analysis Based on Machine Learning LASSO Regression Model”. Journal of Theory and Practice of Engineering Science, vol. 4, no. 01, Jan. 2024, pp. 58-64, doi:10.53469/jtpes.2024.04(01).08. [26] “Enhancing Computer Digital Signal Processing through the Utilization of RNN Sequence Algorithms”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 60-68, https://doi.org/10.62051/ijcsit.v1n1.09. [27] Yu, L., Liu, B., Lin, Q., Zhao, X., & Che, C. (2024). Semantic Similarity Matching for Patent Documents Using Ensemble BERT-related Model and Novel Text Processing Method. arXiv preprint arXiv:2401.06782. [28] Huang, J., Zhao, X., Che, C., Lin, Q., & Liu, B. (2024). Enhancing Essay Scoring with Adversarial Weights Perturbation and Metric-specific AttentionPooling. arXiv preprint arXiv:2401.05433. [29] “Based on Intelligent Advertising Recommendation and Abnormal Advertising Monitoring System in the Field of Machine Learning”. International Journal of Computer Science and Information Technology, vol. 1, no. 1, Dec. 2023, pp. 17- 23, https://doi.org/10.62051/ijcsit.v1n1.03. [30] Yu, Liqiang, et al. “Research on Machine Learning With Algorithms and Development”. Journal of Theory and Practice of Engineering Science, vol. 3, no. 12, Dec. 2023, pp. 7-14, doi:10.53469/jtpes.2023.03(12).02. [31] Huang, J., Zhao, X., Che, C., Lin, Q., & Liu, B. (2024). Enhancing Essay Scoring with Adversarial Weights Perturbation and Metric-specific AttentionPooling. arXiv preprint arXiv:2401.05433.