Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 7, No. 1, 2023 156 Prediction of Cement Slurry Density Based on AMIndRNN Ruohuan Liu, Yazhi Yang, Bin Wang and Xingpeng Zhang School of Intelligent Terminal Industry, Chengdu Technological University, 611730, China Abstract: Cement slurry density is one of the key factors affecting the consolidation effect of oil casing and wellbore, and its value has a direct impact on the quality of cementing and construction safety. In the traditional cementing operation, the professionals perform experiments based on information from adjacent Wells to obtain fuzzy cement density, and then constantly adjust the final cement density in the actual operation, which costs a lot of labor and time, and is not good for real-time operation. In response to the above problems,This paper proposes AMIndRNN(Attention Mechanism combined with Independently Recurrent Neural Network) for cement slurry density prediction, and optimizes IndRNN by introducing SMU activation function. The comparison experiment between AMIndRNN model and baseline model shows that AMIndRNN model has obvious advantages in various performance indexes, which can be used to guide the design of actual cement slurry density. Keywords: Cementing Operation, Cement Slurry Density, Attention Mechanism, Independently Recurrent Neural Network (IndRNN). 1. Introduction Cementing cement slurry density is an important factor affecting the production life and production speed of oil and gas fields, and its value is too high or too low will directly affect the quality of cementing and construction safety[1]. In order to ensure the safe and effective cementing operation in oil field, the accurate prediction of cement slurry density plays a crucial role in improving cementing efficiency and reducing safety accidents. Although a large number of researchers at home and abroad have done many researches on cementing operation, the technical level of cementing operation is still in the primary stage. High cementing cost, low production efficiency and environmental damage[2] are common problems in oil field cementing operations. The successful application of deep learning in agriculture, machinery, biology and other fields provides the possibility for its application in the field of cementing operations. For example, Shang Fuhua et al[3]. applied feedforward neural network to cement bonding quality identification, providing valuable technical support for the engineering application of cement bonding. Therefore, it is of great significance to establish a prediction model of cement density based on deep learning for cement density design in actual cementing operations. Finding the quantitative relationship of complex causality in cement slurry density, realizing scientific cement slurry density prediction, saying goodbye to the experience-based density design method, can not only save labor and time costs, improve cementing quality, but also keep up with the pace of modern science and technology. Based on the above background, this paper proposes a composite model AMIndRNN,which is composed of optimized Independently Recurrent Neural Network ( IndRNN ) and Attention Mechanism ( AM ) to achieve cement slurry density prediction. The innovation of AMIndRNN model is reflected in the introduction of SMU activation function to optimize IndRNN, and the extraction of cementing data features by adding AM. The designed and implemented AMIndRNN model can extract the important features in the input data to the greatest extent. In this paper, the experiment is carried out on the historical cementing data set of a certain offshore oil company. The experimental results show that the error between the predicted value and the actual value is small. The final RMSE value is 0.138 and the MAE value is 0.019, which is better than the baseline neural network model. The method proposed in this paper can be applied to the prediction of cement slurry density, which is of great significance for improving cementing efficiency and reducing cementing cost. 2. Related Work Through the review of the related research results of oil cementing operation, the related research methods in this field can be divided into three categories. 1) Methods based on petroleum engineering expertise. He Yingwei, Yi Hao et al[4]. developed a set of water-soluble resin cement-working fluid system for medium-high pressure, late fracturing, and periodic pressure change of gas storage. Aiming at the problem of annulus pressure in deep shale horizontal wells, Xi Yan et al[5].analyzed the generation and development of micro-annulus under the condition of pre- stressed cementing by means of mechanical experiment and numerical simulation, and clarified the number of fracturing sections of cement sheath under different pre-stress conditions. Luo Jing et al[6].developed a set of systematic and mature large displacement cementing technology for the shallow geological conditions of Bohai Bay, which has achieved remarkable results in field practice and guaranteed the cementing quality of large displacement wells in this block. 2) Methods based on traditional machine learning and statistics. Guan Rongliang[7] proposed to use the least squares support vector machine to predict the cementing quality. The wavelet packet decomposition method is used to extract the formation echo information signal, which is used as the input of the model, and the cementing quality of the sample is used as the output of the sample. Kong Chao[8] uses grey correlation analysis to sort the correlation degree of factors affecting cementing quality, and determines the main 157 influencing factors. The final prediction model is established by combining grey method and fuzzy neural network. The prediction accuracy is high, and it can be used to guide the field cementing construction. Tang Mingyue[9]established a model combining extreme learning machine and K-nearest neighbor to predict the drilling fluid density of high temperature and high pressure wells. The results show that the model has good generalization and fast learning speed. 3) Method based on deep learning. Du Dongnan et al[10].established a cementing quality prediction model based on LM optimized neural network method, which provided technical guidance for on-site cementing quality prediction and cementing construction scheme optimization. Based on the analysis of the importance of cementing engineering and cementing quality, Lyu Heyu[11] established a BP neural network prediction model to predict cementing quality, which provides a new way for its high prediction accuracy and improvement of cementing quality. Deng Yaping and Duan Jiandong et al[12].used cuckoo algorithm to optimize independent recurrent neural network to predict wind power, and the final prediction accuracy was good. Zhao et al[13].proposed a new framework for activity recognition combining short-term spatial / frequency feature extraction and long-term independent recurrent neural networks to solve the time recognition problem of large intra-class distance and small intra-class distance. Li Jia and Huang Zhihao et al[14]. established a model of accurate GDP prediction based on independent recurrent neural networks, which provides great application value for economic forecasting. Wu Zhangyu, Zhu Chengjie et al[15]. used RNN to predict the health status of lithium batteries, and the error of the prediction result was small, no more than 0.02; Wu Lifen et al[16].applied the model combining RNN and LSTM to the machine poetry writing system, and the effect was good. Tong et al[17].established a liquid phase flow measurement method based on LSTM for gas-liquid two-phase flow. The root mean square error is small, and the measurement accuracy and speed are better. 3. Prediction Model of Cement Slurry Density Based on AMIndRNN As shown in Figure 1, the architecture of AMIndRNN model proposed in this paper consists of input layer, IndRNN layer, attention layer and output layer. 固井泥浆数据 X1 X2 X3 Xn输入层 ... IndRNN第一层 IndRNN IndRNN IndRNN IndRNN第二层 IndRNN IndRNN IndRNN IndRNN第L层 IndRNN IndRNN IndRNN .. . .. . .. . .. . IndRNN 层 注意力层 Dense_2 Dense_1 注意力层 输出层 固井水泥浆密度值 Linear ... ... ... ... ... Figure 1. AMIndRNN Structure 3.1. Input Layer The input layer is used to receive the collected cement slurry density data. First, standardize the input data. Due to the different units of each data feature, this section uses the min-max function to perform linear transformation on the data to eliminate the influence of the dimension and order of magnitude of the data. The min-max function formula is shown in (1) . 158 * X -min X max -min (1) where X is the sample data,min is the minimum value of the sample data, andmax is the maximum value of the sample data, *X represents the converted data. Suppose the input data is d iX , d is the characteristic dimension. For n training samples, the input feature tensor d iX can be expressed as: d d d d i 1 2 nX = [X ,X ,...,X ],i = 1,2,...,n . 3.2. IndRNN Layer As one of the traditional deep learning methods, Recurrent Neural Network ( RNN ) is applied to a variety of scenarios, such as text generation, speech recognition, etc. RNN has some limitations due to the problems of gradient disappearance and gradient explosion. As variants of RNN,LSTM and GRU solve the above problems to a certain extent, but in multi-layer neural networks, the disappearance of gradients is still inevitable. Reference [18] proposed the IndRNN model, which can be combined with the unsaturated activation function ReLU to carry out more robust training. The problem of gradient disappearance and gradient explosion is solved to a certain extent by gradient back propagation adjustment[19]. The formula of the model is shown in (2).  �t t t-1h = (Wx +U h +b) (2) Where � represents the hadamard product,W represents the current input weight,U represents the weight vector of 1th , b represents the bias vector, tx represents the input at time t , 1th represents the hidden state at time t - 1 , and  is the activation function. The neural network structure in the model is independent of each other when processing input data, and can realize parallel operation. In addition, the neurons of each layer are also independent of each other, and the output of the hidden state ,n th at time t in the n neuron is shown in formula (3) . , ,n t n t n n t-1 nh = (W X +u h +b ) (3) Where nW represents the input weight of the n layer, nu represents the current weight, and nb brepresents bias. The disadvantage of the ReLU function is that the output is always greater than 0, ignoring the input of negative numbers. In order to better extract the characteristics of cement slurry data, this paper introduces the SMU activation function[20] as the activation function of IndRNN. Its formula is shown in (4). (1 ) (1 ) ( (1 ) ) ( , , ) 2         x xerf x f x x = (4) The optimized model structure is shown in Figure 2. xt-1 xt Layer1 Weight Recurrent+SMU Weight Recurrent+SMU Layer2 Weight Recurrent+SMU Layer3 Ot-1 Weight Recurrent+SMUht-1 Weight Recurrent+SMUht-1 Weight Recurrent+SMUht-1 Ot ··· ··· ··· Figure 2. AMIndRNN structure 3.3. Attention Layer 159 Query Attention Value Key1 Key2 Key3 Key4 Value1 Source Value2 Value3 Value4 Figure 3. Attention mechanism structure The mechanism of human vision, in which humans can quickly filter out valuable information from a large amount of information, provides a source of ideas for the attention mechanism[21], giving different weights to the importance of different inputs[22]. Filtering a small amount of important information from a large amount of information can ignore a lot of irrelevant information. The basic structure is shown in Figure 3. The Transform model uses a complete attention mechanism to replace the traditional structure. Although it is mainly composed of encoders and decoders, the internal mechanism of each encoder is the attention mechanism, and the weight value is not shared between each layer, and each layer is divided into two sub-layers, namely, the feedforward neural network layer and the multi-head self-attention layer. The prediction of cement slurry density can be regarded as a regression problem to be solved. The attention layer is introduced to further filter the feature vectors extracted from the IndRNN layer, which can help the model capture more important high-order data features, thereby strengthening the connection between the layers of data. Assuming that the output of the IndRNN layer is th , its attention weight can be calculated by the following function. exp( ( , )) ( , ) exp( ( , ))   L L t L t t s h q h q = s h q (5) ( , ) i L t tA h = h (6) Where ( , ) Lh q is the attention weight, ( , )tA h is the output of the attention layer, ( , )L ts h q is the scoring function, q is the attention matrix. The scoring function formula is shown in ( 7 ). dot ( , ) ( )L L T t tS h q = h q (7) 3.4. Output Layer The output layer first inputs the feature vector extracted from the attention layer into a fully connected layer, and takes the output of the fully connected layer as the output feature, and then inputs it into the linear layer to output the prediction of cement slurry density. The calculation formula is shown in ( 8 ). ( ) linearf A (8) Among them, the training of the whole model uses Adam optimizer to optimize the MSE loss function. 4. Experiment and Result Analysis 4.1. Experimental Data In this paper, the cementing data from a certain offshore oil company are used. The cement slurry density data from February 2016 to August 2018 are mainly selected, including the size, depth, pressure, circulation temperature, static temperature, temperature gradient, time to the bottom of the well, thickening time, compressive strength, water loss and other characteristics of the well, and the predicted label is density. The specific data description is shown in Table 1. Table 1. Influencing factors of cement slurry density prediction Factor Name Data Value Unit casing size 339.725 mm well depth 700 m bottom hole pressure 1000 psi static temperature 44 ℃ circulation temperature 34 ℃ temperature gradient 4.18 ℃ time to reach the bottom 18 min thickening time 503 min waterloss 1019 gal/sk compressive strength 479 psi 160 4.2. Evaluating Indicator Mean Absolute Error ( MAE ) and Root Mean Squard Error ( RMSE ) are the performance evaluation indexes of the model in this paper. Its specific formula definition is shown in 9 ~ 10. 2 1 1 ˆ( )    n i i i RMSE y y n (9) 1 1 ˆ    n i i i MAE y y n (10) In the formula, iy represents the true value, ˆiy represents the predicted value of the model, and n represents the number of samples. Among them, the higher the prediction accuracy of the model, the smaller the values of RMSE and MAE. 4.3. Evaluating Indicator 4.3.1. Experiment Setting The experimental data is divided into training set and test set according to the ratio of 7 : 3. The training data of the model is the training set, and the prediction accuracy after the model training is tested with the test set. The model parameters in the experiment are shown in Table 2. Table 2. Parameters of each model Name Value Dimensions 10 Batch_Size 300 Learing Rate 0.001 NIndRNN 4 Epochs 200 Activation Function SMU/ReLU In Table 2, Dimensions represents the dimension of the data, Batch _ Size represents the number of data processed in batches during each training, Learing Rate represents the learning rate during the training process, NIndRNN represents the number of layers of IndRNN, Epochs represents the number of iterations, and Activation Function represents the activation function. 4.3.2. Experimental Analysis Firstly, the performance of the optimized IndRNN model is verified. After the experiment of the traditional IndRNN model, the SMU is used to replace the ReLU to conduct the experiment again. The prediction results of two different activation functions are shown in Table 3. The experimental results show that under the same training times, the RMSE and MAE values of the IndRNN model using the SMU activation function are lower. Compared with the traditional IndRNN, its RMSE and MAE are reduced by 12 % and 0.04 %, respectively. This is enough to show that the introduction of SMU activation function can improve the network performance of IndRNN model. Table 3. Comparison of SMU and RELU results Model Name RMSE MAE IndRNN+ReLU 0.36 0.15 IndRNN+SMU 0.24 0.11 Secondly, the feature extraction ability of AMIndRNN model on cement slurry data is verified. For this reason, in the case of the same features, three baseline models RNN, LSTM, and IndRNN were used for comparative experiments. Their predictive analysis results are shown in Table 4. The experimental results show that under the same features and the optimal parameters of each model, the RMSE and MAE values of AMIndRNN are the lowest, and the prediction ability is stronger than that of the baseline model. Table 4. Comparison of the results of each model Model Name RMSE MAE RNN 0.162 0.123 LSTM 0.157 0.112 IndRNN 0.154 0.111 AMIndRNN 0.138 0.019 From table 4, it can be seen that the IndRNN model has lower RMSE and MAE values than RNN and LSTM, and the effect is better. It shows that IndRNN can better extract the potential characteristics of cement slurry density data, and solve the problem of gradient explosion and gradient disappearance of RNN and LSTM, which proves that the model has good performance in the field of oilfield cementing operation. Finally, the RMSE value of AMIndRNN proposed in this paper is 0.054 lower than that of IndRNN, and the MAE value is 0.092 lower, which is more accurate and better. This shows that AMIndRNN can not only solve the gradient problem of traditional RNN and its variant LSTM, but also effectively extract the characteristics of cement slurry input data and 161 improve the prediction accuracy of cement slurry density. 5. Summary In order to solve the demand of cement slurry density prediction in cementing operation, this paper proposes AMIndRNN model for cement slurry density prediction. The experimental results on the historical cementing data of a certain offshore oil company show that the RMSE and MAE values are 0.0014 and 0.11, respectively, which are more competitive than the baseline model. It shows that the model can effectively predict the density of cement slurry accurately. This is of great significance for reducing oilfield cementing time and labor costs. In the follow-up study, we will try to collect more high-quality data, and compare and improve the experiment based on the feature difference of the data, so as to further improve the stability and accuracy of the experimental results. Acknowledgment Research on virtual chemistry experiment teaching platform based on Web3D 2023YB023. References [1] Fan Heng, Liu Meng, Chen Jia, et al. Cementing slurry density monitoring system based on ARM. Modern Electric Technique. Vol. 45 (2022) No. 08, p. 13-17. [2] Yang Jian. Development Status and Prospect of Offshore Oil Cementing Technology. Chemical Industry Management.2020 No.11,p.120-121. [3] Shang Fuhua, Yu Zhidong, Cao Maojun. Application of feedforward neural network in cement bond recognition. Computer Technology and Development. Vol. 23 (2013) No. 09, p. 223-226. [4] He Yingxing, Yi Hao, Li Fei, et al. Performance evaluation of water-soluble resin cementing working fluid system. Oilfield Chemistry. Vol. 38 (2021) No. 04, p. 595-602. [5] Xi Yan, Li Fangyuan, Wang Song, et al. Study on the prevention of cement ring micro-annulus by pre-stressed cementing method. Specialty Reservoir. Vol. 28 (2021) No. 06, p. 144-150. [6] Luo Jing, Wang Hening, Hua Zhiyuan, et al. Cementing Technology for Shallow and Large Displacement Wells in Bohai Bay. Henan Science and Technology. Vol. 40 (2021) No. 30, p. 68-71. [7] Guan Rongliang. Cementing quality prediction based on least squares support vector machine. Xinjiang Oil and Gas. Vol. 14 (2018) No. 04, p. 43-46+3. [8] Kong Chao: Cementing quality prediction and safety evaluation based on gray fuzzy neural network(Master, China University of Petroleum (East China), China 2017).p.12-17. [9] Tang Mingyue. Density prediction of oil-based drilling fluids at high temperatures and pressures - based on adaptive limit learning machine modeling. Xinjiang Oil and Gas. Vol. 15 (2019) No. 02, p. 40-43+3. [10] Du Dongnan, Zheng Shuangjin, He Yingzhuang, et al. Cementing quality prediction method based on LM optimization neural network: taking X block of Shunbei Oilfield as an example. Petroleum Geology and Engineering. Vol. 35 (2021) No. 03, p. 123-126. [11] Lv Heyu. Application of neural network in cementing quality prediction. Oil Drilling Technology. 2002 No. 03, p. 24-26. [12] Deng Yaping, Duan Jiandong, Jia Hao, et al. Optimization of independent recurrent neural network deep learning for ultra- short term wind power prediction based on Cuckoo algorithm. Power grids and clean energy. Vol. 37 (2021) No. 09, p. 18-26. [13] Zhao Beidi, Li, Shuai , Gao, Yanbo, et al. A Framework of Combining Short-Term Spatial/Frequency Feature Extraction and Long-Term IndRNN for Activity Recognition. Sensors (Basel, Switzerland). Vol. 20 (2020) No. 23, p. 6984. [14] Li Jia, Huang Zhihao, WANG Jiahui. GDP forecasting based on independent recurrent neural network approach. Statistics and Decision-Making. Vol. 36 (2020) No. 14, p. 24-28. [15] Wu Zhangyu, Zhu Chengjie, Wang Mingyan. RNN-based lithium battery health prediction. Green Technology. Vol. 23 (2021) No. 18, p. 201-203. [16] Wu Lifen, Yan Xueyong, Zhao Ji. Research on Machine Poetry Composition Based on RNN Models and LSTM Models. Science Technology Innovation and Application. Vol. 11 (2021) No. 27, p. 48-50. [17] Tong Weiguo, Zeng Shichao, Li Zhixiang, et al. LSTM-based liquid-phase flow measurement for gas-liquid two-phase flow. Instrumentation Technology and Sensors. 2021 No. 11, p. 94- 98. [18] Li Shuai, Li Wanqing, Cook Chris, et al. Independently Recurrent Neural Network (Indrnn): Building A Longer And Deeper Rnn. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).p. 5457-5466. [19] D Stursa, P Dolezel. Comparison of ReLU and linear saturated activation functions in neural network for universal approximation. 2019 22nd International Conference on Process Control (PC19). 2019,p. 146-151. [20] Biswas Koushik, Kumar Sandeep, Banerjee Shilpak, et al. SMU: smooth activation function for deep networks using smoothing maximum technique. [21] Shi Lei, Wang Yi, Cheng Ying, et al. A Review of Research on Attention Mechanisms in Natural Language Processing. Data Analysis and Knowledge Discovery. Vol. 4 (2020) No. 05, p. 1-14. [22] Kaiyou Song, Hua Yang, Zhouping Yin. Multi-Scale Attention Deep Neural Network for Fast Accurate Object Detection. IEEE Trans. Circuits Syst. Video Techn. Vol. 29 (2019) No. 10, p. 2972-2985.