Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 4, No. 2, 2023 1 Bearing Fault Diagnosis based on Residual Network Attention Mechanism Chunxiu Huang School of Yangtze University, Jingzhou, Hubei, 434000, China Abstract: Aiming at the problem that the detection of bearing fault diagnosis is rarely applied in the research of image classification, a new method based on residual network and attention mechanism is proposed to identify bearing fault diagnosis. One-dimensional vibration signals are transformed into two-dimensional time-frequency images by continuous wavelet transform (CWT), which are input into the model for classification. In order to solve the problem that the traditional convolutional neural network model ignores the low diagnostic accuracy of channel attention and spatial attention due to the loss of important features, the attention mechanism CBAM module is added to make up for the loss of channel features and spatial features in the traditional model. At the same time, the residual network Resnet combined with the attention mechanism can better capture the global information of the time frequency graph, and make up for the defects of the residual network module. The experimental results show that the model has high diagnostic accuracy in rolling bearing fault diagnosis, which proves that the proposed method is effective and feasible. Keywords: Residual Network; Attention Mechanism; Bearing Failure; CWT. 1. Introduction Rolling bearing is an important component of modern industrial large machinery, a wide variety of bearings, the application field is also very wide. Most of the mechanical failures, most of them are caused by bearing failure, while the failure of rolling bearings is unavoidable, once the failure, may lead to mechanical system failure and unexpected safety problems, such as huge economic losses and casualties. Under normal circumstances, when the bearing is in the early stage of failure, the fault is not very obvious that the mechanical equipment can still operate normally, but if the bearing diagnosis is not handled in advance, it will have serious consequences. Therefore, early bearing fault diagnosis is very important in industrial production. In recent years, deep learning technology has been widely studied in rolling bearing fault diagnosis and achieved good results. More and more deep learning methods are applied in the field of mechanical fault diagnosis. At the same time, many scholars have successfully completed various diagnostic tasks by using machine learning methods. For example, Shao Haidong proposed A novel method for intelligent fault diagnosis of rolling bearings using ensemble deep auto-encoders[1]. A series of autoencoders with different characteristics are designed to get rid of the dependence on artificial feature extraction and expert experience, and the vibration signal reduces the human intervention. The experimental results show that the method greatly improves the efficiency of rolling bearing fault diagnosis. Liu et al. [2] proposed a fault diagnosis model of transfer learning based on Wasserstein-generated adversarial networks with deep full convolution conditions, and mapped class labels to source domain data through a matrix, which not only enhanced the supervision of the learning process, but also enhanced the effect of class domain alignment. Chen et al. [3] put forward wavelet analysis and verified the essence of inner product operation of wavelet transform through simulation and field experiments. Zhang et al. [4] propose A new deep learning model for fault diagno sis with good antinoise and domain adaptation ability on raw vibration signals, which uses the original vibration signal as input (using data enhancement to generate more input) and uses the wide kernel of the first convolution layer to extract features and suppress high- frequency noise. In order to improve the domain adaptability of the model, AdaBN is introduced to solve the problem that the accuracy of fault diagnosis used by CNN is not high. Li et al. [5] proposed a fault diagnosis method based on short-time Fourier Transform (STFT) and convolutional neural network (CNN) to realize end-to-end fault pattern recognition. The vibration signal of rolling bearing is divided into training set and test set by short-time Fourier transform. Then the training set is entered into the CNN, and the CNN is learned and the parameters are updated. Finally, the updated CNN model is applied to the test set, and the result of fault identification is output. The method has high identification accuracy for different types of faults. These models have achieved good results in the field of bearing fault diagnosis. However, for the problem of noise interference, this paper proposes to add a Convolutional Block Attention Module (CBAM) with channel attention and spatial attention. CBAM module can serialize the attention feature map information in both channel and space dimensions, and then multiply the two-feature map information with the original input feature map for adaptive feature correction to generate a better feature map. Meanwhile, CBAM has a good image classification effect in processing image problems. Based on this, the one- dimensional signal is processed by continuous wavelet transform (CWT), and the signal recognition of Case Western Storage bearing fault is converted into the image recognition of the time frequency for classification. In order to alleviate the problem of noise interference, the residual network module combined with attention mechanism can better capture the global information of time frequency graph, and make up for the defects of the residual network module. Compared with traditional fault diagnosis methods, the above methods still have great improvement in detection accuracy, model training convergence speed and detection of bearing 2 vibration signals mixed with noise. 2. Related Work 2.1. Resnet With the deepening of the number of layers of convolutional neural networks, the fitting ability can be improved, but in practical applications, the number of layers of the network will decrease, and even gradient disappearance or explosion problem will occur. To solve this problem, He Keming proposed deep residual network (ResNet) in 2015 [6], which avoids the defect of gradient disappearance or gradient explosion in neural networks with the deepening of network layers. At present, ResNet has been widely used in image processing. The residual network structure is shown in Figure 1. Fig 1. ResNet residual structure The structure of ResNet is shown in the figure. The input of the neural network is x, F(x) is the residual function, and the output is H(x)=F(x)+x; The residual network can be connected with the input and output of different layers through the cross-layer connection structure, so that the network degradation problem can be alleviated. Such a structure is of great benefit in backpropagation because the cross-layer connected structure can propagate the gradient losslessly and does not cause the gradient to disappear. Therefore, adding several identity maps to the neural network increases the network depth without causing network degradation. 2.2. CBAM The Convolutional Block Attention Module (CBAM) [7] integrates the spatial attention module and the channel attention module. Attention mechanism not only improves the efficiency of image classification, but also enhances the ability of network to extract effective information and extract more important information. The convolutional attention mechanism diagram is shown in Figure 2 below. Fig 2. Convolutional Block Attention Module The channel attention module first compacts the feature map in spatial dimension, adopts average pooling and global maximum pooling in step S, and models channel correlation through two fully connected layers and corresponding activation functions in step E, and combines the output to obtain the weight of each feature channel. After the weights of the feature channels are obtained, the original features are weighted to each channel by multiplication to complete the re-calibration of the original features in the channel dimension. The spatial attention module firstly performs global average pooling and global maximum pooling based on channels respectively. All input channels are pooled into two real numbers, and two (h*w*1) feature graphs are obtained from the (h*w*c) shaped inputs. Then a 7*7 convolution kernel is used to form a new (h*w*1) feature map. Finally, the output feature map and the input feature map of the spatial attention module are integrated to get the final feature. 3. Experiment and Analysis 3.1. Experiment Procedure 3.1.1. Data Sources and Background This paper adopts the rolling bearing experimental data set [8] from the Bearing Data Center website of Case Western Reserve University in the United States, and takes the experimental data of type 6205 deep bearing SKF with driving end as the research object, including the data under four loads of 0hp, 1hp, 2hp and 3hp. The sampling frequency is 12 kHz and the bearing speed is 1 797 r/min. The fault types are divided into inner ring fault, outer ring fault, rolling element fault and normal state. The test set and the training set are divided according to 6:4, including 3600 training samples and 2400 test samples. Detailed data description is shown in Table 1 below. Table 1. Fault sample information tag Fault location Fault size Number of training sets Number of test sets 0 Inner ring fault 0.007 360 240 1 Inner ring fault 0.014 360 240 2 Inner ring fault 0.021 360 240 3 Outer ring fault 0.007 360 240 4 Outer ring fault 0.014 360 240 5 Outer ring fault 0.021 360 240 6 Rolling element fault 0.007 360 240 7 Rolling element fault 0.014 360 240 8 Rolling element fault 0.021 360 240 9 Normal state 0 360 240 3.1.2. Analysis of Experimental Results The data processing and experiments in this paper are all implemented under the Windows10 system, the programming language is Python, and the development tool is PyCharm. In order to ensure the same sequence of input data each time, set the number of random seeds to 0, set the epoch size to 30 times, that is, 30 iterations, and set the batch_size size to 5, 3 that is, select 5 samples for training each time. The diagnostic accuracy of model training is shown in Figure 3 below. With the increase of iterations, the diagnostic rate of the model is continuously improved. The accuracy of the training set reaches 100% after the second round of iteration, and the accuracy of the test set reaches about 99.7%. It can be seen from the figure that the method proposed in this paper has a good classification effect in the classification and recognition of bearing defects. The loss function curve of model training can be seen in Figure 4, from which we can see that the cross-entropy loss function value of training set and test set is also close to 0 and tends to be stable after the second round of iteration. Fig 3. Diagnostic accuracy curve of model training Fig 4. Loss function curve of model training The training in FIG. 3 and FIG. 4 shows that the proposed method can diagnose the bearing fault type of the test bench more accurately, and verifies that the model proposed in this paper has better adaptability under different working conditions and can accurately diagnose the bearing fault type. 4. Conclusion In this experiment, one-dimensional vibration signals of Case Western Reserve are transformed into two-dimensional time-frequency images by continuous wavelet transform and input into the model for classification. Aiming at the problem that the traditional convolutional neural network model ignores channel attention and spatial attention, resulting in low diagnostic accuracy due to the loss of important features, this paper proposes a rolling bearing fault diagnosis method based on residual network and attention mechanism. The CBAM module of attention mechanism is added to the model to make up for the loss of channel and spatial features in the traditional model. At the same time, CBAM has a good image classification effect in processing image problems. Resnet combined with the attention mechanism can capture the global information of time frequency graph better and make up for the defect of residual network module. The experimental results show that the model has high diagnostic accuracy in rolling bearing fault diagnosis. References [1] SHAO H,JIANG H,LIN Y,et al.A novel method for intelligent fault diagnosis of rolling bearings using ensemble deep auto- encoders [J]. Mechanical Systems and Signal Processing, 2018, 102: 278-297. [2] LIU Y Z,SHI K M,LI Z X,et al.Transfer learning method for bearing fault diagnosis based on fully convolutional conditional wasserstein adversarial networks [J]. Measurement, 2021, 180: 553-563. [3] Chen J, Li Z,Pan J.Wavelet transform based on inner product in fault diagnosis of rotating machinery:a review[J]. Mechanical Systems and Signal Processing,2016(70-71):1-35. [4] Zhang W, Peng G L,Li C H.A new deep learning model for fault diagno sis with good antinoise and domain adaptation ability on raw vibration signals[J].Sensors,2017,17(2):425. [5] Li Heng,Zhang Qing,Qin Xian-rong.Fault diagnosis method for rolling bearings based on short-time Fourier transform and convolution neural network[J].Journal of Vibration and Shock, 2018, 37(19):124-131. [6] He K,Zhang X Y,Ren S Q.Deep Residual Learning for Image Recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2016. [7] Niu Ruihua, Yang Jun, Xing Lianxin, et al. Micro expression recognition algorithm based on convolutional attention module and dual channel network [J]. Computer applications, 2021,41 (09): 2552-2559. [8] CHANG R Y,PODGUＲSKI A,YANG J.Case Western Reserve Univ.,Cleveland,OH[J].Software Engineering IEEE Transactions on, 2008,34: 579-596.