Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 9, No. 2, 2024 19 Research and Practice of Computer Application Technology in Big Data Analysis Jiliang Bi Xihua University, Chengdu 610039 China Abstract: This article aims to discuss the application of computer application technology in the field of BD (big data) analysis and the challenges it faces, and put forward corresponding countermeasures. Firstly, this article introduces the definition, characteristics and theoretical basis of BD analysis. Then, the specific application of computer application technology in BD analysis is elaborated in detail. Through case analysis, the practical application effect and value of computer application technology in BD analysis are demonstrated. The research results show that computer application technology plays a vital role in BD analysis, which can improve the efficiency of data processing, the accuracy of analysis and the scientific decision-making. Moreover, the research also reveals the technical, methodological and practical challenges faced in the process of BD analysis. In view of these challenges, this article puts forward corresponding countermeasures and suggestions, including strengthening the construction of policies and regulations, promoting technological innovation and research and development, and promoting Industry-University-Research's cooperative cooperation. These countermeasures have certain guiding significance and practical value for promoting the development of BD analysis. Keywords: Computer application technology; BD analysis; Data mining. 1. Introduction As information technology rapidly advances, BD has emerged as a pivotal characteristic and invaluable asset of the modern era [1]. BD is notable for its substantial volume, diverse variety, swift processing speeds, and relatively low value density, posing challenges for traditional data processing and analysis techniques [2]. Computer application technology, as the mainstay of information processing, plays a crucial role in BD analytics [3]. Through this technology, BD can be efficiently collected, stored, managed, and analyzed, unveiling valuable insights that inform decision- making and drive innovation and progress across various industries [4]. Given this backdrop, exploring the application of computer application technology in BD analysis is highly significant from both theoretical and practical perspectives [5]. This study aims to delve into the research and practical applications of computer application technology in BD analysis. Theoretically, this investigation can enhance understanding of the interplay between computer application technology and BD analysis, contributing to the refinement of related theoretical frameworks. Practically, it can guide the implementation of BD analytics, enhancing data processing efficiency and accuracy, and facilitating the broader adoption of BD in diverse domains. 2. Fundamentals of Computer Application Technology 2.1. The development of computer application technology Computer application technology has undergone significant evolution and diversification since its inception in the mid-20th century. Initially, its primary focus was confined to numerical computations and basic data processing tasks, such as those related to scientific calculations and engineering designs [6]. However, with the relentless advancement of computer hardware and the concurrent surge in software technology, the scope of computer applications has steadily broadened to encompass information management, automatic control, artificial intelligence, and numerous other domains. Notably, the advent of cutting-edge technologies in the 21st century, including the Internet, cloud computing, and the Internet of Things, has propelled computer application technology's pervasive integration into virtually all societal sectors, emerging as a pivotal driver of societal progress and development. 2.2. The core concept of computer application technology The core concepts of computer application technology include computer system, computer software, computer network and database. Computer system is the infrastructure to realize computer application, including hardware system and software system [7]. Computer software is the key tool to realize computer application, including system software and application software. Computer network is an important way to realize information transmission and resource sharing between computers. Database is an important tool for storing, managing and retrieving data, which provides basic support for BD analysis. 2.3. Correlation between computer application technology and BD analysis There exists a tight interconnection between computer application technology and BD analysis. Firstly, computer application technology offers indispensable technical backing for BD analysis [8]. Its efficient operational capabilities facilitate swift processing and analysis of large datasets. Additionally, computer software's data processing and analytical functions enable the extraction of valuable insights buried within the data. Furthermore, the information transmission and resource-sharing abilities of computer networks permit cross-regional and cross-domain data integration, fostering collaborative analysis. The database's 20 data storage and management proficiency also ensures the precision and dependability of BD analysis. Conversely, BD analysis has spurred ongoing advancements and innovations in computer application technology. Given BD's unique traits, conventional computer application techniques struggle to meet its analytical demands. This necessitates continual refinement and evolution of computer application technology to tackle BD analysis challenges. For instance, there's a growing need for more efficient data acquisition and preprocessing techniques to enhance data quality and usability. Similarly, developing more sophisticated data storage and management systems is crucial for guaranteeing data security and accessibility. Moreover, designing more intelligent data analysis and mining algorithms is essential for boosting analytical precision and efficiency. Lastly, creating more intuitive data visualization and reporting tools is paramount for effective result presentation and analysis. 3. Theoretical Framework of BD Analysis 3.1. Theoretical basis of BD analysis BD contains a large amount of data that traditional software tools find difficult to capture, manage, and process in a reasonable amount of time. This data originates from various sources, spanning social media platforms, e-commerce transactions, Internet of Things devices, log files, and more, accumulating at a remarkable rate [9]. What sets BD apart is not merely its sheer size, but also its diverse and intricate nature, encompassing both structured and unstructured forms of data. The analysis of BD relies heavily on the principles of statistics, DM (data mining), machine learning, and database technology. Statistics offer a structured approach to data collection, organization, analysis, and interpretation. DM is dedicated to unearthing concealed patterns, associations, and trends buried within vast datasets. Machine learning enables computers to learn autonomously and enhance their performance through algorithmic means, thereby facilitating intelligent data analysis. Database technology serves as a dependable and efficient medium for data storage, retrieval, and management. The seamless integration and application of these theoretical pillars constitute the cornerstone of BD analysis, empowering the extraction of invaluable insights from gargantuan datasets. Figure 1 illustrates the DM process. Figure 1. DM process 3.2. The process and method of BD analysis The process of BD analysis usually includes four stages: data collection, preprocessing, analysis modeling and result interpretation. In the data collection stage, it is needed to determine the data source and adopt appropriate technical means to collect it; In the pretreatment stage, the data are cleaned, integrated and transformed to ensure the quality and consistency of the data; In the stage of analysis and modeling, statistics, DM and machine learning are used to deeply analyze the data and mine potential information and knowledge; In the result interpretation stage, the analysis results are presented in an intuitive and easy-to-understand way to support decision-making. BD analysis methods are shown in Table 1. Table 1. BD analysis method Analysis type Specific method Description and examples Descriptive analysis Data visualization Display the data in the form of charts, images, etc., which is convenient for intuitive understanding of data distribution and laws. Numerical statement Show the basic statistical information of data, such as average and standard deviation, by summarizing and calculating. Predictive analysis Regression analysis Use the relationship between independent variables and dependent variables to predict the future trend of dependent variables. Time series analysis Analyze the data in chronological order and predict the future development trend. Normative analysis Optimization algorithm Find the optimal solution through mathematical methods, such as linear programming and genetic algorithm. Decision tree Show the decision-making process through tree diagram to help decision makers choose the best scheme. 4. The Application of Computer Application Technology in BD Analysis In the process of BD analysis, computer application technology plays a vital role. From data collection and pretreatment to storage and management, to analysis and mining, and finally to visualization and reporting, computer application technology provides strong support. Data acquisition and preprocessing technology Data collection is the starting point of BD analysis, and its quality directly affects the accuracy of subsequent analysis. Computer application technology has realized the comprehensive collection of structured, semi-structured and unstructured data by means of network crawler, sensor and 21 log capture. Moreover, with the help of data cleaning, deduplication, conversion and other preprocessing technologies, the quality and availability of data can be further improved, laying a good foundation for the subsequent analysis work. Data storage and management technology When confronted with vast amounts of data, efficiently and securely storing and managing it poses a significant challenge. Fortunately, computer application technology offers solutions like distributed storage systems and data warehouses, which enable dependable storage and streamlined management of large datasets. Additionally, techniques like data backup, recovery, and encryption safeguard the security and integrity of the data. Data analysis and mining technology At the heart of BD analysis lies data analysis and mining. Computer application technology equips analysts with a range of data analysis tools and algorithms, including statistical analysis, machine learning, and deep learning. These tools empower analysts to extract invaluable insights and knowledge from the depths of massive datasets. Data visualization and reporting technology Data visualization and reporting is an important means to present the analysis results in an intuitive and easy-to- understand way. Computer application technology provides a wealth of data visualization tools and report generators, such as charts, dashboards, data maps, etc., which makes the analysis results more intuitive and easy to understand. Moreover, through interactive visualization technology, real- time interaction with data can be realized, and the efficiency and accuracy of decision-making can be improved. 5. Case Analysis and Practical Application 5.1. Case study: the specific application of computer application technology in BD analysis Taking the e-commerce industry as an example, the use of computer application technology in BD analysis is reflected in many aspects. In the data collection and preprocessing stage, the e-commerce platform collects massive user data through log files, user behavior tracking and other technologies, and uses data cleaning and conversion technologies to process these data for subsequent analysis. In the stage of data storage and management, distributed storage system and data warehouse are used to store and manage these large-scale data. In the stage of data analysis and mining, machine learning algorithms are used for tasks such as user portrait construction and product recommendation. Finally, in the stage of data visualization and reporting, the analysis results are displayed in the form of charts and dashboards through visualization tools, which provides intuitive data support for decision makers. 5.2. Practical application: current situation and challenges of industrial application In terms of industrial applications, BD analysis has penetrated into many fields such as finance, medical care, education and logistics. For example, in the financial field, BD analysis is used for risk assessment, fraud detection, customer relationship management, etc. In the medical field, it is used for disease prediction, personalized treatment and optimization of medical resources. However, with the deepening of application, various industries are also facing some challenges. These challenges include data security, privacy protection and lack of technical talents. In addition, because the data characteristics and analysis requirements of different industries are quite different, how to choose appropriate technologies and methods according to specific needs has also become a problem to be solved. 5.3. Effect evaluation: the effect and value of technology application The effect evaluation of computer application technology in BD analysis can be carried out from multiple dimensions. The first is the technical evaluation, including the efficiency of data processing and the accuracy of analysis. Secondly, it is the evaluation of business level, such as whether BD analysis has improved the revenue of enterprises, reduced costs or optimized business processes. Finally, it is the social assessment, such as whether BD analysis has played an active role in promoting social progress and improving people's livelihood. On the whole, the application of computer application technology in BD analysis has produced remarkable effects and values, but there is still room and potential for further improvement. 6. Challenges and Countermeasures of Computer Application Technology in BD Analysis 6.1. Challenges With the wide application of computer application technology in BD analysis, a series of technical and practical challenges have gradually emerged. These challenges not only come from the technology itself, but also involve methods, practices and policies. See Table 2 for details. Table 2. BD analysis method Type of challenge Specific challenges Description and problems Technical challenge Data security and privacy protection There are data security challenges in the process of BD analysis, because the scale and complexity of BD make traditional encryption and protection technologies difficult to deal with, and the improvement of data value increases the risk of leakage and abuse. Method challenge Algorithm optimization and model construction BD analysis faces the challenge of algorithm and model selection. Because of the expansion and complexity of data, traditional methods are difficult to adapt, and the diversity and dynamics of BD also increase the difficulty of building effective models. Practical challenges Cross-field cooperation and talent training BD analysis needs cross-domain cooperation, but domain differences and communication barriers make cooperation difficult, and the lack of BD analysis talents with both technology and business in the market restricts its development. 22 6.2. Countermeasure and suggestion Given the aforementioned challenges, we can propose tailored countermeasures and suggestions encompassing policy, regulatory, and technological development perspectives. Initially, it's imperative for the government to establish pertinent policies and regulations that govern the collection, storage, and utilization of BD, thereby safeguarding data security and privacy. Furthermore, fostering cross-disciplinary collaboration and personnel training should be prioritized to cultivate an enabling environment conducive to BD analysis. Subsequently, enterprises must ramp up their investments in technology research and development, continually refining algorithms and models to elevate the precision and efficiency of BD analysis. Simultaneously, emphasis should be placed on the innovation and deployment of data security and privacy-preserving technologies to facilitate seamless BD analysis. Lastly, the active participation of all societal sectors in BD analysis practices is crucial to collectively advance the development and application of BD technology. Through the harmonious collaboration between political entities, industries, universities, and research institutions, we can collectively tackle the challenges posed by computer application technology in BD analysis and promote the healthy progression of the BD industry. 7. Conclusions In this study, a comprehensive examination of computer application technology's utilization within BD analysis is presented. The discussion spans from data acquisition and preprocessing to storage, management, analysis, mining, and visual reporting. Case studies illuminate the distinct application methods and benefits of this technology in BD analysis. Furthermore, the research identifies technical, methodological, and practical hurdles encountered during BD analysis and offers tailored solutions and recommendations. The study's primary contribution lies in systematically outlining the application procedures and pivotal technologies of computer application technology within BD analysis, serving as a valuable reference for related research and practical applications. The case studies demonstrate the real- world effectiveness and value of computer application technology in BD analysis, thereby enhancing the study's practical relevance. Additionally, the identification of key challenges in BD analysis and the provision of targeted strategies and suggestions offer guidance for advancing the field. Moreover, this research informs the development of policies and regulations by the government, providing a foundation for creating a conducive environment for BD growth. Overall, the study contributes to a deeper understanding of the role of computer application technology in BD analysis and its broader implications for related fields and policies. References [1] Yao Hairui. Application of BD analysis and mining technology in marketing [J]. Computer Science and Artificial Intelligence, 2023, 1(4):24-27. [2] Chen Zhitai. Application analysis of BD mining technology in enterprise ERP [J]. Business News, 2020, No.193(03):106-107. [3] Shi Tingting, Liu Weihua, Liu Shuangyin, et al. Research on the application of DBSCAN optimization algorithm in experimental text BD analysis [J]. Computer Science and Application, 2020, 10(5):8. [4] Jin Jubo, Ge Lei, Xu Xiuli. Development and application of computer data mining technology [J]. Journal of Zhejiang College of Water Resources and Hydropower, 2019, 031 (006): 68-72. [5] Pan Xinyu, Zhang Xiaomiao. Pollutant tracking and forecasting algorithm based on BD analysis [J]. Computer and Digital Engineering, 2023, 51(5):1096-1100. [6] Wang Guojie, Yu Jiantao, Wang Liping, et al. Research on the algorithm platform of geo-temporal BD [J]. Industrial Control Computer, 2023, 36(6):113-114. [7] Shang Yunfeng. Research on computer information processing technology and application under the background of BD [J]. Computer fan, 2019, 000(002):231. [8] Li Yueen, Han Xinzhi. Research on product design method based on image data mining [J]. Design, 2023, 8(4):6. [9] Ding Weijie, Liang Ronghua, Sun Guodao, et al. Research progress of crime data visualization [J]. Journal of Computer Aided Design and Graphics, 2023, 35(7):979-989.