Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 4, No. 3, 2023 121 Research on Urban Monitoring Network Management Platform based on AI Data Stream Processing Qinyu Song China Agricultural University, Beijing, China Abstract: In recent years, with the upsurge of the continuous development of high-tech information industry, the security industry has also made a new leap in technology, products, industry and so on. The traditional single small-scale monitoring and management software obviously cannot meet the actual needs of users in terms of system scale and functional requirements. This paper puts forward a network management platform for urban monitoring based on AI data stream processing. With the help of comprehensive processing and analysis of urban complex information, through network remote monitoring means, information supervision and public opinion trends are used to control social environmental security, timely warning and response to unexpected accidents, so as to ensure that governance departments can fully control and prevent possible problems such as road traffic, population distribution and disaster emergency. Keywords: AI Data Stream Processing; Urban Monitoring Network Management. 1. Introduction Urban network monitoring and management system is an important embodiment to measure the modern management level of a city, and it is the basis to realize the security and stability of a safe city and even the whole country [1]. The existing network monitoring network is usually a self- contained system of various industries, communities and enterprises, but the monitoring demand has developed into wide-area urban networking, and the front-end video equipment has reached hundreds of thousands of levels [2]. Large-scale information sharing and urban management demand promote the development of monitoring system to network management platform. The rise of AI data stream processing stems from the interaction of many factors. First of all, with the rapid development of Internet and Internet of Things technology, the emergence of big data has become a trend [3]. Sensors, social media, logs and other data sources are constantly emerging, showing the characteristics of high dimension, high speed and diversity. This massive real-time data is a challenge to the traditional batch processing method, which requires a more efficient processing method. At the same time, major breakthroughs in the fields of machine learning and deep learning enable AI algorithm to efficiently analyze and model large-scale data. This provides a new solution for real- time data stream processing. By applying AI algorithm to real-time data stream processing, real-time analysis, prediction and decision of data can be realized, thus improving the response speed and accuracy of the system [4]. In addition, with the continuous improvement of computing power and the progress of chip technology, modern embedded systems can run complex AI algorithms on smaller devices. This enables AI to be embedded in edge devices and sensors to realize local processing and analysis of real-time data streams. This distributed data processing method has lower delay and better privacy protection, and is suitable for scenes that need rapid response and data security protection. To sum up, the rise of AI data stream processing is the result of the interaction between the generation and demand of big data, the progress of AI algorithm and the development of embedded system technology [5]. This processing method can help people better understand and use real-time data, and provide more intelligent and efficient solutions in various fields. It not only provides support for real-time decision- making, but also brings new opportunities and challenges to the development of intelligent transportation, smart cities, industrial automation and other fields. The management platform of urban monitoring network based on AI data stream processing is research to improve the management efficiency and intelligent level of urban monitoring network by using artificial intelligence and data stream processing technology [6]. The platform can obtain useful information and make corresponding decisions by analyzing and processing the data collected by various monitoring devices (such as cameras and sensors) in the city in real time. It can improve the management level of urban monitoring network through intelligent and efficient data processing and analysis, and provide strong support for urban safety and governance. 2. The Current Situation of 2.AI Data Stream Processing Technology and Urban Monitoring Network Management Platform 2.1. AI Data Stream Processing Technology In recent years, a new data mode-data stream has appeared in many application fields. Different from the tables stored statically in the traditional database model, the data in this model is no longer a permanent relational form, but a large number of continuous, fast and time-varying data streams, such as financial data, sensor data, network traffic data and so on [7]. Compared with the database model, the data flow model has the following characteristics: the data elements in the stream arrive online; The system cannot control the order of newly arrived data to be processed; Once an element in the data stream is processed, it is either discarded or archived [8]. In view of these characteristics, a strategy model for dealing with continuous data streams is proposed, namely continuous query. In this model, data arrive in the form of continuous addition, which drives the continuous execution of registration query. In addition, due to the continuous addition 122 of streams, all data cannot be read into memory at one time, and the execution of blocking operators (such as various aggregate functions) must be unblocked in some way [9]. AI data stream processing technology refers to a technology that applies artificial intelligence (AI) algorithm to the processing and analysis of real-time data streams [10]. It combines technologies in the fields of big data processing, machine learning and real-time decision-making, aiming at realizing real-time analysis, prediction and decision-making of high-speed, high-dimensional and diverse data streams. Through intelligent algorithms and models, AI data stream processing technology can help enterprises and organizations to better understand and utilize real-time data and make immediate responses and decisions in a rapidly changing environment [11]. AI data stream processing technology has been widely used in many fields, such as financial fraud detection, intelligent traffic management, data analysis of Internet of Things, online advertising optimization, etc. It can help people make better use of a large number of real-time data generated rapidly, and make decisions and processes automatically, so as to improve work efficiency and decision accuracy. With the continuous development of technology, AI data stream processing technology will continue to evolve and innovate, bringing more applications and challenges to various industries. 2.2. The Status of Urban Monitoring Network Management Platform Urban monitoring network management includes real-time monitoring and discovery of various security events such as port scanning, dos attacks and large-scale worms [12]. The traditional solution is to use intrusion detection technology (including misuse detection and anomaly detection), combined with various methods of worm discovery and network traffic flow analysis to complete the alarm and prevention of security incidents. The development of Internet provides convenience for the efficient sharing of resources and information on a global scale, but it also poses new challenges to network security and intrusion detection systems. The increasingly complex network system structure, widely used distributed application environment, mass storage and high-bandwidth transmission technology make the traditional intrusion detection more and more unable to meet the security requirements of the system. The urban monitoring network management platform not only improves the level of urban safety management and intelligence, but also faces some shortcomings and challenges. Among them, privacy issues and data security risks are the most prominent, and effective privacy protection and security measures need to be taken to deal with them. In addition, there are also some problems, such as non-uniform technical standards, emphasizing technology while ignoring social factors, challenges of operation and maintenance costs, false alarms and low public participation. However, these problems can be solved or alleviated by formulating clear privacy policies, strengthening security protection measures, promoting the unification of technical standards, and strengthening social participation and public education, so as to realize the sustainable development and good application of urban monitoring network management platform. In the future, with the further development of technology and society, the urban monitoring network management platform is expected to be gradually improved, providing safer and smarter management and services for the city, and balancing the relationship between data processing and personal privacy protection to meet the needs and expectations of society and the public for the construction of smart cities. In this case, we need to propose and implement new intrusion detection methods from system model, architecture, implementation technology and other aspects to meet the increasingly complex needs of urban monitoring network management. 3. Urban Monitoring Network Management Platform based on AI Data Stream Processing The urban monitoring network management platform not only improves the level of urban safety management and intelligence, but also faces some shortcomings and challenges. Through research, we put forward an urban monitoring network management platform based on AI data stream processing, which combines technologies in the fields of big data processing, machine learning and real-time decision- making, aiming at realizing real-time analysis, prediction and decision-making of high-speed, high-dimensional and diverse data streams. Through intelligent algorithms and models, AI data stream processing technology can help enterprises and organizations to better understand and utilize real-time data and make immediate responses and decisions in a rapidly changing environment. The main steps of AI data stream processing technology include real-time data collection and cleaning, feature extraction and dimensionality reduction, model training and updating, real-time decision-making and response, visualization and reporting. Through these steps, real-time data streams can be accurately analyzed, classified, clustered and predicted, thus providing users with insight and decision support about real-time data. This technology is widely used in many fields, such as financial fraud detection, intelligent traffic management, data analysis of Internet of Things, online advertising optimization and so on. It can help people make better use of a large number of real-time data generated rapidly, and make decisions and processes automatically, so as to improve work efficiency and decision accuracy. With the continuous development of technology, AI data stream processing technology will continue to evolve and innovate, bringing more applications and challenges to various industries. But at the same time, we need to pay attention to privacy protection and data security to ensure compliance and security in data processing. Through continuous research and development, AI data stream processing technology is expected to make a greater breakthrough in real-time data analysis and decision-making, and create a smarter and more efficient working and living environment for people. Many security applications are typical data stream applications, especially the network security event monitoring and analysis system. These security applications must deal with online and continuous high-speed network data flow without delay, and all network data cannot be stored in external memory. Our research is based on continuous query model and sliding window technology, using data stream management system as the platform for stream data processing, and applying it to urban network security monitoring system. In this system, the data flow management platform performs a large number of continuous queries registered in the system, filters, connects and aggregates continuous network flows, and completes the monitoring and alarm of various security incidents, thus effectively 123 supporting the requirements of real-time, accuracy and flexibility of the monitoring system. The structure of urban monitoring network management platform system based on AI data stream processing is shown in Figure 1. Figure 1. System Structure Diagram The input data source of the urban monitoring network management platform based on AI data stream processing is the network packets collected by network packet detectors distributed at multiple collection points through packet capture. The original network packet is preprocessed to form a data stream that conforms to the format. The system defines three streams in advance. Through these three streams, most network security event monitoring functions can be completed. The specific stream format is discussed in the following data preprocessing module. If users need streams of other specific applications, they can register according to the corresponding stream format. Various monitoring requirements are continuous queries written in CQL language, such as intrusion detection statements similar to IDS attack rules, worm detection rules statements, and statistics maintenance statements corresponding to various related network monitoring. The city monitoring network management platform based on AI data stream processing will act on the continuous incoming data stream and produce the query results in continuous stream form. The query results can be forwarded to a trigger module in real time to alarm the detected intrusion behavior, or cached in the system and submitted to users for viewing when necessary. The processing plan of urban monitoring network management platform based on AI data stream processing covers intelligent transportation system, security video monitoring, security bayonet monitoring, electronic police system, etc., which is combined with intelligent video analysis to improve the security standards of smart cities, combine vehicle-mounted and personal mobile video monitoring, cover urban video monitoring to all important locations and regions, and use comprehensive video management platform to summarize existing monitoring resources and all social communication video resources. Using the same video surveillance information management platform to provide sharing and application of traffic police, public, security, urban management and municipal video resources improves the visualization, intelligence and control of the whole system. The overall synthesis of urban monitoring network management platform is shown in Figure 2. Figure 2. Overall synthetic diagram of urban monitoring network management platform 4. Conclusion AI data stream processing plays an important role in urban monitoring network management platform. It collects, processes and analyzes a large number of video and image data in real time, and uses machine learning and intelligent algorithms to realize functions such as target recognition, anomaly detection, data aggregation and intelligent analysis. This can help improve the intelligent level of urban safety management and resource scheduling. The network security event monitoring system based on data flow management platform has an accurate, concise and complete interface language and powerful expressive ability, which can integrate various functions of rule-based IDS, such as network attack detection, worm discovery and network status monitoring, and has good scalability. It is a promising research idea to apply data stream processing technology to network security. In the next step, we will consider how to further optimize the query engine of the platform, including multi-query optimization through query grouping and indexing, so as to further improve the execution efficiency of the query engine and thus improve the efficiency and accuracy of network security event monitoring. References [1] Zhong, Zijia,Kim,et al.Low-Cost and Energy-Saving Wireless Sensor Network for Real-Time Urban Mobility Monitoring System[J].Journal of Sensors, 2015.DOI: 10.1155/ 2015/ 685786. [2] Fernández-Lozano J.J,Martín-Guzmán Miguel,Martín-ávila Juan,et al.A Wireless Sensor Network for Urban Traffic Characterization and Trend Monitoring[J].Sensors, 2015, 15 (10):26143-26169.DOI:10.3390/s151026143. [3] Oh B K, Park H S.Urban safety network for long-term structural health monitoring of buildings using convolutional neural network[J].Automation in construction, 2022(May): 137. DOI: 10.1016/j.autcon.2022.104225. [4] Segura-Garcia J, Felici-Castell S, Perez-Solano J J,et al.Low- Cost Alternatives for Urban Noise Nuisance Monitoring Using Wireless Sensor Networks[J].Sensors Journal IEEE, 2015, 15(2):836-844.DOI: 10.1109/ JSEN. 2014. 2356342. [5] Kouichi H, Ngae P, Kumar P,et al.An optimization for reducing the size of an existing urban-like monitoring network for retrieving an unknown point source emission[J].Geoscientific Model Development, 2019(8).DOI:10.5194/GMD-12-3687- 2019. [6] Li W, Shimin L, Jingfeng Y,et al.Dynamic Traffic Congestion Simulation and Dissipation Control Based on Traffic Flow 124 Theory Model and Neural Network Data Calibration Algorithm [J]. Complexity, 2017, 2017:1-11.DOI:10.1155/2017/5067145. [7] Lin B.Research on Data Release and Location Monitoring Technology of Sensor Network Based on Internet of Things[J].Journal of web engineering, 2021(3):20. [8] Khan S, Nazir S,Iván García-Magario,et al.Deep learning- based urban big data fusion in smart cities: Towards traffic monitoring and flow-preserving fusion[J].Computers & Electrical Engineering, 2021, 89:106906. DOI: 10.1016/ j. compeleceng. 2020.106906. [9] Chen X, Wan X, Ding F,et al.Data-Driven Prediction System of Dynamic People-Flow in Large Urban Network Using Cellular Probe Data[J].Journal of Advanced Transportation, 2019, 2019(PT.1):95-106. [10] Chen X, Wan X, Ding F,et al.Data-Driven Prediction System of Dynamic People-Flow in Large Urban Network Using Cellular Probe Data[J].Journal of Advanced Transportation, 2019.DOI:10.1155/2019/9401630. [11] Hale G.Gathering Big Data analytics through network monitoring[J].Control Engineering, 2017, 64(3):36-37. [12] Hale G.Using wireless to gain a network and data monitoring edge[J].Control Engineering, 2016, 63(12):42-43.