Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 5, No. 3, 2023 41 Research on Method of Creating Dynamic Weld of ROI Region Based on Faster‐RCNN Qiang Song, Chenlei Zhao, Shenghong Wu, Xi Xu Geely University of China, Sichuan, 641423, China Abstract: Aiming at the issues of weld marking noise in welding path planning of the third generation welding robot, that the creation of ROI region is employed as the approach to noise suppression. However, traditional ROI region construction methods can only create ROI regions at a fixed location by presetting parameters in the system. The welding target position usually produces displacement in the control range of the tolerance due to an important tolerance concept in the welding process, which may result in an ROI region created with traditional methods is not able to coincide with the ROI region required by the system, thereby affecting the quality of the welding. To improve the location accuracy of the created ROI region, a dynamic ROI region creation method based on Faster-RCNN target detection algorithm was proposed. Experimental results show that this method effectively reduce weld marking noise. Keywords: Faster-RCNN, Weld dynamic ROI area, Weld marking. 1. Introduction In the process of using Forstner operator to quickly mark weld features, a large number of feature points in non-weld areas will be extracted by Forstner algorithm on account of those reasons including object construction and camera shooting environment, resulting in lot of noise interference to the system. As shown in Fig. 1a, the yellow part refers the marked feature points, and a large number of non-target noise points are marked in the non-weld area. These noise feature points will not only increase the operating burden of the system, but also interfere with the system's welding path extraction work, and ultimately affect the welding robot's judgment of the welding path. a b Figure 1. Comparison of noise effects before and after the construction of ROI region The creation of ROI area of weld , narrowing the target area of feature extraction of Forstner operator is thus used to solve the problem of system noise caused by this feature marking process.As shown in the red box in Fig.1b, the area where the weld is located is separated and cut out by the target detection algorithm and created as the ROI area of the weld, thus further limiting the extraction area of feature points and reducing noise. 2. Object Detection Method Based on Deep Learning There are mainly two kinds of object detection methods based on deep learning: one is object detection method based on regression; and the other is target detection algorithm based on candidate region. With the former is of excellent performance in the real-time detection of the target, though the detection accuracy is not as good as the latter, reflected in the performance difference especially for the small target encountered in the multi-target detection. The deep learning object detection algorithm based on the candidate region is also known as the two-step method. Compared with the aforementioned deep learning object detection method based on regression, the construction process of image feature sample box is increased. Therefore, the detection accuracy of this method is obviously higher, but it also makes the algorithm more complex, directly affects its detection efficiency, and the real-time performance of the algorithm is not strong. At present, typical representatives of this method include Faster-RCNN, Mask-RCNN and other algorithms, among which the use of Faster-RCNN algorithm to solve practical engineering problems is a hot research and application direction in the field of target detection. The Faster-RCNN algorithm extracts the necessary features by inputting the image into the appropriate CNN network model. The feature is input into RPN network to generate the recommended candidate region, and the feature region extracted from CNN network model is input into pooling layer to generate the feature region with a fixed size. Finally, the targets are differentiated and detected at the full connection layer, which is used to classify and regression the detected targets. Therefore, the Faster-RCNN algorithm has two key points: CNN network model and RPN network. CNN network model: CNN network model is one of the key factors that determine the operation effect of deep learning object detection algorithm. From the earliest LeNet5 network model to the revolutionary AlexNet network model, the detection accuracy of CNN network model has been continuously improved by deepening the depth and complexity of network model. So that when ResNet network model was developed, its detection accuracy even exceeded the level of human brain cognition, reaching 96.43%. The Faster-RCNN algorithm in this paper is based on AlexNet network model, which is an efficient and reliable one. After its appearance, the traditional target detection algorithm is no longer applicable. New object detection algorithms with high efficiency and high precision based on deep learning and 42 convolution training by a large number of images are emerging constantly. AlexNet network model structure is as follows: Figure 2. AlexNet network model structure It can be seen from the figure that AlexNet network model is deeper and more complex than LeNet5 network model, so it is more accurate. Image feature detection can be completed after the input image passes through 5 convolution layers and 3 fully connected layers. At the same time, due to the improvement of the traditional activation function and the introduction of ReLu activation function, the convergence and implementation speed of the network model have been greatly improved. RPN network (regional suggestion network) : Faster- RCNN algorithm can be regarded as the construction of RPN network (Fig.3) improved and optimized on the basis of Fast- RCNN algorithm, with two main functions included : training Fast-RCNN network and optimizing configuration of Fast- RCNN network by sharing weights. The RPN network model inputs the features processed by CNN model into sliding window and performing convolution to obtain 256- dimensional feature vectors, which are entered into the full connection layer to obtain scores (target score and background score) and displacement (4 offset relative to the original coordinate). Finally, the candidate areas are solved based on these results. Figure 3. RPN network model structure 3. Construct the Experimental Platform Considering that the research object is in an absolute static state during the working process, the requirement for real- time performance of the system is not high;Therefore, the Faster-RCNN algorithm, an approach to guarantee the welding accuracy, was selected to carry out the verification test based on AlexNet network model on Matlab2018b platform.The system configuration is as follows: Table 1. System configuration Software Matlab2018b System Win10 (64 bit) CPU I5-6500 RAM 16G (ddr4) GPU GTX 950 (2G) NVIDIA GPU Computing Toolkit CUDA 10.2 4. Experimental Data Set Construction In the actual welding operation environment (take automobile welding line as an example), in order to ensure production efficiency, the working state of each welding robot unit is only to weld several specific welding spots or welds, with single welding type and high repetition. It can be seen that in the actual deployment process of welding robot, it is necessary to train the welding ROI region detection system. The system only needs to realize the construction of training data by changing the camera shooting Angle and position according to the actual conditions of welding parts. The advantage of this is that high detection rates can be achieved with a small amount of training data. In short, there are two main reasons for this: (1) The operating environment of welding robot is stable and does not involve the problem of target detection in different scenes. (2) Detection image and training image acquisition objects 43 are consistent, which does not involve detection object classification. Therefore, the research object in this paper is simpler and requires fewer training data set pictures. In this paper, the types of welds are divided into 3 groups (Straight line, arc and composite weld are simulated respectively), with 10 pieces in each group and a total of 30 pieces for training(Fig.4 a). There are 40 images in each test set and 120 images in total for testing. The training set and part of the test set (Fig.4 b) are enumerated: a b Figure 4. Data set enumeration 5. Faster-RCNN Data Training The completed training set image collection is followed by the ROI labeling on the welding target area individually , and then save these ROI annotation areas to the work area after the annotation is complete. After image preprocessing is completed, training parameters of the Faster-RCNN algorithm need to be preset. The main parameters are shown in the following table: Table 2. Preset parameters of Faster-RCNN algorithm MiniBatchSize 1 MaxEpochs 20 InitialLearnRate 1e-4 Solvername sgdm LearnRateSchedule piecewise 6. Experimental Result Further test set is used to test the effectiveness of this method for welding targets in the completion training of the Faster-RCNN algorithm model .The current evaluation of the algorithm's effectiveness is mainly based on some indicators, including two indicators: recall rate and accuracy rate; Recall rate refers to the ratio of the number of images with welding target detected to the number of samples in the test set images. Accuracy refers to the ratio between the number of correct target images and the total number of detected target images. Figure 5 lists two typical detection defects that affect accuracy: a. Error areas are detected; b. The detection position deviates from the target area. a b Figure 5. List of detection defects that affect detection accuracy Table 3. Faster-RCNN detection effect test and analysis results recall rate accuracy rate Straight weld 100% 100% Arc weld 100% 65% Composite weld 100% 78% mean value 100% 81% It can be seen from Table 3 that the average recall rate of the research object mentioned in this paper is 100%, which results in the realization of the 100% detection ration of the ROI area of the weld in the detection set constructed in this paper. However, the accuracy of weld ROI region detection was low, with an average of 81% (especially for arc welds). Based on the AlexNet network model, its detection accuracy is generally 57.1%, and the average accuracy in the ROI area of the weld studied in this paper reaches a relatively high level of 81%. Therefore, for the research object described in this paper, the detection effect is good, and can effectively reduce the noise level of welding system. Acknowledgment Supported by university-level scientific research project of Geely University of China "Research on Welding Path Planning Method Based on Binocular Machine Vision" (2022xxsy003). References [1] Zhang J , Tao D . FAMED-Net: A Fast and Accurate Multi- scale End-to-end Dehazing Network[J]. IEEE Transactions on Image Processing, 2019. [2] Chenfan Liu, Junqi Shen, Shengsun Hu, Dingyong Wu, Chao Zhang, Hui Yang,Seam tracking system based on laser vision and CGAN for robotic multi-layer and multi-pass MAG welding[J].Engineering Applications of Artificial Intelligence,Volume 116,2022. [3] Shengfeng Chen, Dezhi Yang, Jian Liu, Qi Tian, Feitao Zhou.Automatic weld type classification, tacked spot recognition and weld ROI determination for robotic welding based on modified YOLOv5[J].Robotics and Computer- Integrated Manufacturing,Volume 81,2023. [4] Yusen Geng, Min Lai, Xincheng Tian, Xiaolong Xu, Yong Jiang, Yuankai Zhang,A novel seam extraction and path planning method for robotic welding of medium-thickness 44 plate structural parts based on 3D vision[J].Robotics and Computer-Integrated Manufacturing,Volume 79,2023. [5] Yanbiao Zou, Guohao Zeng,Light-weight segmentation network based on SOLOv2 for weld seam feature extraction[J]. Measureent, Volume 208,2023. [6] Liangyuan Deng, Ting Lei, Chaoqun Wu, Yibo Liu, Shiyu Cao, Song Zhao,A weld seam feature real-time extraction method of three typical welds based on target detection[J]. Measurement, Volume 207,2023. [7] Runquan Xiao, Yanling Xu, Zhen Hou, Chao Chen, Shanben Chen,A feature extraction algorithm based on improved Snake model for multi-pass seam tracking in robotic arc welding [J]. Journal of Manufacturing Processes,Volume 72,2021. [8] Runquan Xiao, Yanling Xu, Zhen Hou, Chao Chen, Shanben Chen,An adaptive feature extraction algorithm for multiple typical seam tracking based on vision sensor in robotic arc welding[J].Sensors and Actuators A: Physical, Volume 297, 2019. [9] Hui Wang, Yu Huang, Guojun Zhang, Youmin Rong,A novel method for dense point cloud reconstruction and weld seam detection for tubesheet welding robot[J].Optics & Laser Technology,Volume 163,2023. [10] Jianyong Wang, Chunyang Mu, Song Mu, Rui Zhu, Hua Yu,Welding seam detection and location: Deep learning network-based approach[J].International Journal of Pressure Vessels and Piping,Volume 202,202