Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 4, No. 1, 2023 17 An Underwater Target Ranging System based on Binocular Vision Yian Wang * Northeast Petroleum University, Daqing, China * Corresponding author Email: llswya@163.com Abstract: In order to accurately measure the distance and size information of underwater targets, an underwater target ranging and size measurement system based on binocular vision is proposed. Underwater binocular calibration method is used to obtain the underwater parameters of binocular camera, and the underwater images are calibrated to make them coplanar and aligned. The improved CLAHE algorithm is used to process underwater images and improve the image contrast. The parallax map is obtained by SGBM stereo matching algorithm, and the distance and size of the target are measured and calculated according to the parallax map. The results of underwater experiment show that the measurement accuracy of the system is high for the distance and size of the close-range underwater target, and the measurement error conforms to the requirements of underwater experiment. Keywords: Underwater Binocular Vision; Underwater Calibration; Underwater Image Processing; Vision Measurement. 1. Introduction With the continuous development of society and economy, countries are constantly seeking new resources. The ocean accounts for more than 70% of the earth's total area and contains a large number of biological, mineral and other resources, which has become the object pursued by various countries [1]. No matter the development of Marine resources, or underwater operations such as deep-sea exploration [2], underwater pipeline detection [3], salvage and rescue [4], etc., are inseparable from underwater target location and size measurement. Compared with radar, infrared, laser and other sensors are expensive and require high deployment conditions. Underwater optical vision is characterized by high resolution, and the introduction of binocular vision technology can accurately measure close-range targets [5], and reduce the use cost. The uneven illumination in underwater environment, coupled with the influence of water on the absorption, scattering and refraction of light, will greatly reduce the image quality. Therefore, in view of the underwater environment and other problems, this paper processed the obtained underwater images by improved CLAHE image processing algorithm, and obtained the internal and external parameters of the underwater camera by underwater calibration method. Based on this, a binocular stereo vision system is designed and implemented, which is suitable for underwater target ranging and size measurement. 2. Underwater Ranging System based on Binocular Vision The overall process of underwater target ranging and size measurement based on binocular vision is shown in Figure 1. Firstly, the improved CLAHE algorithm is applied to the underwater image processing. In this paper, the left and right checkerboard images collected underwater are calibrated by the improved Zhang's calibration method, and the underwater parameters of the binocular camera are obtained, including rotation matrix, translation vector, etc. Image processing was carried out for the input underwater images, and binocular correction was carried out by using the parameters after underwater calibration. The stereo matching algorithm of SGBM is used to match the corrected underwater images, and then the underwater parallax map is obtained. Finally, the distance and size of the target are measured according to the principle of binocular distance measurement and the space two-point distance formula. Underwater left checkerboard image Underwater right checkerboard image Image processing Image processing Single targeting Single targeting Binocular calibration Internal and external parameters Underwater image of left camera Underwater image of left camera Binocular correction SGBM stereo matching Measure distance and dimensions Use the mouse to select points Image processing Image processing Figure 1. Flow chart of underwater target ranging system based on binocular vision 3. Underwater Binocular Imaging Model and Underwater Calibration Method For underwater shooting, the camera container shell needs to be sealed and waterproof, and transparent glass cover plate is usually used in front of the container. As a result, light is refracted twice as it travels from underwater objects to the camera lens, at the interface between water and glass, and at the interface between glass and air. Since the glass is thin and homogeneous, and the target distance is much larger than its thickness, the propagation process of light in the glass can be ignored [6]. The simplified underwater imaging model of the camera is shown in Figure 2.  , , WW W P X Y Z in Figure 2 is the target point to be measured; 1 O is the real position of the camera; 2 O is the intersection point of the extension line of incident light in water on the optical axis of the camera, which is taken as the virtual camera position; O is the refraction point of incident light on the interface between water and air; 18 A is the intersection point of camera optical axis and the interface between water and air; 1  and 2  are respectively the incidence Angle and the refraction Angle. Z θ1 θ2 O2 O1 O A P(XW , YW , ZW) Camera Camera_V Air Water Figure 2. Underwater imaging model 1 n and 2 n are the refractive indices of water and air respectively. According to the triangle similarity and refraction law 1 1 2 2 sin sinn n  , it can be obtained:  1 1 1 1 2 2 2 2 2 1 tan sin tan sin O O n O O n            Because the optical axis of the real camera coincides with that of the virtual camera, the projection length of OA on the two cameras is the same, that is, 1 2 X X , where 1 X and 2 X are the imaging lengths of OA in the real camera and the virtual camera respectively. Therefore:  1 1 1 X f OA OO     2 2 2 X f OA OO    According to formula 2 and 3, we can get:  1 1 2 2 2 1 f nOO OOf n     Formula 4 shows that the focal length of the camera when used underwater will change with the refractive index of water and air. Suppose that the coordinate of point P in the world coordinate system is ( , , ) w w w X Y Z , point u P is the projection of point P in the pixel coordinate system, and its coordinate in the pixel coordinate system is ( , )u v , then the transformation formula of point P from the world coordinate system to the pixel coordinate system is[7]:  0 0 1 0 0 0 0 1 1 0 0 0 0 0 1 1 0 0 1 0 0 0 1 1 w w w u Xdxu f R T Y v v f ZZ dy                                               Where, dx and dy are the physical length of unit pixel along the X and Y axes[8], Z represents the depth of point P ,   0 0 ,u v is the coordinate of the origin of image coordinate system in pixel coordinate system, f is camera focal length, R is rotation matrix, and T is translation vector. When the camera is placed underwater, due to the influence of water on light scattering, the refraction of incident light in the water at the front cover glass of the camera, and the distortion of imaging caused by the camera itself, the underwater imaging of the camera will have relatively serious distortion. In this paper, Zhang's calibration method is improved, and the first three distortion coefficients of the camera radial distortion polynomial are obtained through the high-order polynomial function, so as to eliminate the distortion problem of the camera underwater imaging better and ensure the measurement accuracy of the system. (1) As the distance between the imaging point and the image center increases, the radial distortion increases correspondingly, so the quadratic and higher-order polynomial functions related to the distance are used to correct the radial distortion [9]. Radial distortion correction formula is as follows:  2 4 6 1 2 3 2 4 6 1 2 3 (1 ) (1 ) r r x x k r k r k r y y k r k r k r              In the formula, ( , )x y is the coordinate of the distortion point in the image before correction; ( , ) r r x y is the coordinate of the distortion point in the image after correction; r is the distance from the distortion point to the image center, and 1 k , 2 k and 3 k are the radial distortion coefficients. (2) The optical axis of the lens is not perpendicular to the image plane, causing tangential distortion. Tangential distortion correction formula is as follows:  2 2 1 2 2 2 1 2 2 ( 2 ) ( 2 ) 2 t t x x p xy p r x y y p r y p xy              In the formula, 1 p and 2 p are tangential distortion coefficients. In order to obtain the coordinates ( , , ) w w w X Y Z of point P in the world coordinate system, the value of Z needs to be calculated. In an ideal situation, binocular ranging is to obtain the left and right pictures of the target respectively from two identical and parallel cameras, so as to calculate the target distance value [10]. The principle diagram of binocular ranging is shown in Figure 3: In the figure, P is the coordinate point of the three-dimensional space to be measured; l O and r O are the optical center of the left and right cameras; T is the optical center distance of the left and right cameras, namely the baseline length; l P and r P are the coordinates of point P in the image coordinate system of the left and right cameras; the horizontal coordinates of pixel points of point P in the left and right images are l X and r X respectively; Z is the vertical distance between point P and the camera. 19 P T Z Xl Xr Pl Pr Ol Or f f Figure 3. Binocular visual ranging principle As can be seen from Figure 3, l r l r PPP POO ∽ . According to the triangle similarity relationship [11]:  ( )T X X Z fl r T Z       The expression of distance Z between the target to be measured and the camera can be summarized as follows:  fT fT Z X X dl r      In the formula, d is the parallax between the left and right cameras, l r d X X  ; T are known. Thus, the depth Z of point P can be found simply by obtaining parallax d . 4. Improved Limited Contrast Adaptive Histogram Equalization Image Processing Method The main feature of the limited contrast adaptive histogram equalization algorithm (CLAHE) is that it limits the contrast [12], overcomes the problem of excessive amplification noise and blocky effect, and mainly enhances the local contrast, thus enhancing the image details [13]. Common CLAHE algorithm enhances R, G and B components respectively in RGB color space, and then combines them into RGB images [14]. In this paper, the improved CLAHE algorithm is used for color space conversion on the obtained underwater left and right eye images, converting the input RGB space into LAB space. Due to the separation of brightness components in LAB space, it is easier to process the image and improve the operational efficiency. Finally, RGB synthesis is carried out. The improved CLAHE algorithm is used to process the underwater image, as shown in Figure 4. It is obvious that the processed image has enhanced the details, improved the contrast, and significantly improved the visual effect. (a) (b) Figure 4. Improved CLAHE algorithm seabed scene processing effect((a) Original view of submarine scenery ;(b) Improved CLAHE algorithm handles graphs) 5. Underwater Experiment 5.1. Underwater Experiment Software and Hardware Environment The hardware used in the experiment includes: experimental water tank, laptop and MSK-SM binocular camera, as shown in Figure 5. The experimental tank size is 1m×1m×1m, and the actual water discharge depth is 0.8m. The configuration of the notebook is as follows: CPU is AMD 5800H; The memory is 32GB; The GPU is NVIDIA GeForce RTX 3070. Underwater binocular camera configuration: MSK-SM binocular camera, image resolution 640×480, baseline length 25mm. The software environment is windows 10, MATLAB 2021b, OpenCV. 5.2. Underwater Calibration Image Acquisition In this paper, improved Zhang's calibration method is used to calibrate binocular camera underwater. GP400 standard checkerboard calibration board with specification of 12*9 was adopted, and the size of each square was 30mm*30mm. The calibration board was shown in Figure 5. The underwater binocular camera and checkerboard calibration board were placed underwater, and the checkerboard images of the underwater left and right cameras with different positions were shot. Among them, 16 pairs of clear underwater calibration maps with a total of 32 were selected for underwater calibration. Figure 5. Experimental environment and checkerboard calibration board 5.3. Improved Limited Contrast Adaptive Histogram Equalization Image Processing Method Figure 6. Improved CLAHE algorithm for image processing contrast graph The improved CLAHE algorithm is used to process the acquired calibration images and underwater object images, 20 and the processing results are shown in Figure 6. The processing results show that the color and contrast of the underwater image after processing are enhanced obviously, which can effectively represent the object features. 5.4. Calibration and Correction of Underwater Binocular Camera Using MATLAB, 13 pairs of checkerboard images were read in turn, checkerboard corner points were extracted from the input images, and the left and right underwater cameras were calibrated respectively. After corner points were extracted, the parameters of the corresponding monocular camera were obtained through calculation. The corner error distribution diagram of a single camera was obtained, as shown in Figure 7(a), and the relative position diagram between a single camera and the checkerboard calibration board was shown in Figure 7(b). It can be seen from Figure 7(a) that most of the scattered points are clustered in the interval [-0.1, 0.1], with high precision. (a) (b) (c) Figure 7. Underwater calibration effect diagram((a) Corner error distribution diagram ;(b) Position diagram of a single camera relative to the calibration plate ;(c) The relative position of binocular camera and calibration plate) According to the above analysis, the underwater refraction makes the underwater image distortion more serious. Therefore, the first three distortion coefficients of the radial distortion correction polynomial and the first two distortion coefficients of the tangential distortion correction polynomial are obtained by using the high-order polynomial for calibration calculation in this paper. The underwater calibration parameters of the binocular camera are shown in Table 1. Finally, the relative position diagram of binocular camera and calibration plate was obtained, as shown in Figure 7(c). Table 1. Calibration parameters of underwater binocular camera Parameter Left camera Right camera Internal parameter matrix 441.9644 316.4981 441.58 0 0 0 0 39 234.520 1          440.2659 323.7030 439.92 0 0 0 12 225.5039 10          Distortion coefficient  0.7512 0.4994 0.0001 0.0014 0.5704  0.7524 0.5419 0.0017 0.0015 0.6471  Rotation matrix 0.9999 0.0002 0.0037 0.0002 0.9999 0.0013 0.0037 0.0013 0.9999             Translation vector  24.9695 1.1318 1.4443  The underwater image was corrected by using the internal and external parameters obtained from underwater calibration, so that the left and right eye underwater images were coplanar and aligned. The comparison of underwater images before and after correction is shown in Figure 8. (a) Underwater image before correction (b) Corrected underwater image Figure 8. Underwater image before and after correction 5.5. Underwater Distance Measurement Experiment The images of underwater objects are obtained by binocular underwater camera, and the improved CLAHE algorithm is used to process the images. The SGBM algorithm is used for stereo matching and the depth map of the target at different distances is obtained. The binocular ranging principle is used to measure the distance of the target in the way of selecting points with the mouse. The actual underwater target ranging effect is shown in Figure 9. Figure 9. Underwater target ranging actual effect Images of underwater targets were collected by the binocular camera at a distance of 0.30m, 0.40m, 0.50m, 0.60m, 0.70m and 0.8m. The depth of different positions is shown in Figure 10: (a) 0.30m (b) 0.40m (c) 0.50m (d) 0.60m (e) 0.70m (f) 0.80m Figure 10. Parallax map of target at different distances The average value of multiple measurements is taken. The results obtained from the underwater target ranging 21 experiment of the system are shown in Table 2 below Table 2. Experimental results of underwater target ranging True distance (m) Experimental ranging value (m) Absolute error (m) Relative error (%) 0.300 0.3036 0.0036 1.20 0.400 0.4079 0.0079 1.98 0.500 0.5120 0.0120 2.40 0.600 0.5781 0.0219 3.65 0.700 0.6662 0.0338 4.83 0.800 0.7536 0.0464 5.80 According to the experimental results in Table 2, when the measured distance is within 0.5m, the error between the measured value of the target and the real distance between the target and the camera is small, the error is less than 2.4%, and the ranging effect is ideal. When the measured distance is greater than 0.5m, the error between the measured distance value of the target and the real distance value of the camera increases gradually, but the error is less than 6%, which meets the requirements of underwater experiment. 5.6. Underwater Dimensional Measurement Experiment The target images are collected and processed successively at 0.2m, 0.30m, 0.40m, 0.50m, 0.60m and 0.70m from the distance camera. SGBM algorithm is used for stereo matching and the depth map of the target at different distances is obtained. The dimensions between the selected two points on the target are obtained by using the binocular ranging principle and the distance formula. The known actual diameter of the target is 0.21m, and the measurement effect of the actual underwater target size is shown in Figure 11. Figure 11. Actual rendering of underwater target size measurement Images of underwater targets are collected at 0.20m, 0.30m, 0.40m, 0.50m, 0.60m and 0.7m away from the underwater camera. The depth of different positions is shown in Figure 12. (a) 0.20m (b) 0.30m (c) 0.40m (d) 0.5m (e) 0.6m (f) 0.8m Figure 12. Parallax map of target at different distances At different positions, two points around the edge of the target diameter are selected to calculate its size value. The average value of multiple measurements is the measurement value of the underwater binocular size of the system. The experimental results of the measurement of the underwater target size are shown in Table 3 below. Table 3. Experimental results of size measurement of underwater target Measuring distance (m) Binocular size measurement(m) Absolute error (m) Relative error (%) 0.20 0.2129 0.0029 1.38 0.30 0.2136 0.0036 1.71 0.40 0.2132 0.0032 1.52 0.50 0.2041 0.0059 2.81 0.60 0.2027 0.0073 3.48 0.70 0.1997 0.0103 4.90 According to the experimental results in Table 3, when the measurement distance is less than 0.5m, the system has a small error between the measured value and the actual size of the underwater target, which is less than 2.81%, indicating a high accuracy. When the measurement distance is greater than 0.5m, the error between the measured value and the actual size of the underwater target obtained by the system gradually increases, but the error is less than 6%, which meets the requirements of underwater experiment 6. Conclusion In view of the technical requirements of underwater binocular vision target ranging and size measurement, a binocular vision based underwater target ranging and size measurement system is designed. The system uses underwater binocular proximity for underwater target image acquisition. Through improved CLAHE algorithm, the obtained underwater image is processed and the image contrast is improved. The first three distortion coefficients of the radial distortion correction polynomial and the first two distortion coefficients of the tangential distortion correction polynomial are obtained by using the improved Zhang's calibration method to calculate the camera underwater calibration. The left and right eye underwater images are corrected in stereo by using the internal and external parameters of the camera underwater calibration, so that the underwater images meet the requirements of coplanar and line alignment. SGBM stereo matching algorithm is used to obtain the parallax map of the corrected underwater image, and then the underwater target ranging and size measurement are completed. The results of underwater experiments show that the underwater images processed by the improved CLAHE algorithm can enhance the details, improve the contrast, and greatly improve the visual effect. In the distance of 0.5m, the average accuracy of the system measurement is 97.19%, which has a high accuracy. When the measurement distance is greater than 0.5m, due to the scattering and refraction of light in the underwater environment, and the increase of distance, there will be more interference of invalid pixels in the imaging plane, which increases the difficulty of binocular stereo matching, decreases the matching accuracy, and leads to the corresponding increase of measurement error. However, the measurement error of the system is still within 6%, which meets the requirements of underwater experiment. 22 References [1] Shuai Peng. Application of Underwater Robot in offshore petroleum Engineering [J]. Petroleum and Chemical Equipment, 2019, 22(05):63-65. [2] Li Xiaowei, Ge Xiyun, Feng Xuelei, Chen Nanruo. Development of Deep-sea Exploration target tracking System Based on Image Recognition [J]. Mechanical Engineering & Automation,2018(03):172-174. [3] Huang Shuling. Research on Underwater Target Detection and Tracking Method of AUV [D]. Harbin Engineering University, 2014. Yao Bin, Yang Lingzhi, Yu Jiuzheng, Ji Zhenning, Hu Gaixing. Research and application of digital stratified water flooding technique based on Wave Code Communication [J]. China Petroleum Machinery, 20, 48 (05): 71-77. [4] Lionetto L, Casolla B, Mastropietri F, et al. Application research of 3D imaging sonar system in salvage process[J]. Applied Mechanics and Materials, 2014, 643(8): 279-282. [5] Zhuang Sufeng, Tu Dawei, Zhang Xu, Yao Qinzhou. Binocular stereo vision underwater corresponding points matching and 3 d reconstruction method research [J]. Journal of instruments and meters, 2022 lancet (5): 147-154. The DOI: 10.19650 / j. carol carroll nki cjsi. J2209215. [6] Wei Jingyang. Underwater high-precision 3D Reconstruction Method based on Binocular stereo vision [D]. Harbin Institute of Technology,2017. [7] Yang Weimin, Yang Guangxi, Sun Shuang. Application of camera calibration technology in oilfield monitoring equipment [J]. Information Systems Engineering, 2020(05): 90-91. [8] Liu Yangyang. Research on Binocular Vision Target Detection and Ranging Method of UAV [D]. Chongqing University, 2019. [9] Dong Peng, Zhou Feng, Zhao, Yafei Wang, Zetian Mi, and Xianping Fu. Automatic size measurement method of underwater sea cucumber based on binocular vision [J]. Computer Engineering and Applications,2021,57(08):271-278. [10] Li Ke, Wu Tao, Liu Qingqing. Human Contour Extraction Based on depth map and Improved Canny Algorithm [J]. Computer Technology and Development, 2021, 31(05): 67-72. [11] Wang Mingji, Chen Qiumeng, Ren Fushen. Target distance measurement system based on binocular vision [J]. Automation and instrumentation, 2022 (7): 5-8. DOI: 10.14016 / j. carol carroll nki. 1001-9227.2022.07.005. [12] Li Mingjie, Liu Xiaofei. Underwater image enhancement algorithm based on CLAHE study [J]. Journal of the science and technology, 2014 (10): 93. DOI: 10.19392 / j. carol carroll nki. 1671-7341.2014.10.075. [13] Yu Yide, Zhou Manli, Wang Hongping. Underwater image enhancement based on limited contrast color correction [J]. Radio Engineering,2017,47(09):16-20. [14] Cui Qianqian, Li Ruiyang. Research on Defogging algorithm based on CLAHE color image [J]. Science and Technology Innovation,2019(33):56-57.