Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 8, No. 3, 2024 84 Research on a Machine Vision‐based Blade Measurement Method Zheng Zou, Peixuan Zhu, Tianli Fu School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan Hubei, China Abstract: A high-precision visual measurement method for the geometric dimensions of oil pump blades is proposed, utilizing an auxiliary measurement mechanism to assist in sub-pixel edge positioning. Based on images of calibration boards, the camera distortion is corrected by partitioning the image into regions and establishing a mapping index between pixel coordinates and real-world coordinates. Under fixed object and image distances, images of the oil pump blades and the contact between the blades and the pump are separately captured. The Canny operator and Sigmoid function are employed to fit the edges, extracting sub-pixel coordinates, which are then mapped to real-world coordinates. Finally, a least squares edge fitting is conducted to compute the dimensional parameters. Experimental results indicate that the use of auxiliary measurement mechanisms effectively enhances the precision of blade measurements, as compared to measurements solely based on oil pump blade images and images of the blade in contact with the pump. Keywords: Machine Vision; Oil Pump Blades; Dimension Measurement; Sub-pixel. 1. Introduction The project focuses on the geometric dimensions of oil pump blades, as depicted in Figure 1. The required measurements include the length of the blade, specified as 24.90 ± 0.03 mm, and the width, specified as 11.962 ± 0.024 mm. In today's rapidly advancing industrial automation, the production of oil pump blades has already achieved automated assembly line manufacturing. With advancements in mechanical part production and manufacturing processes, the requirements for produced components have escalated from millimeter-level precision to micron-level precision. Consequently, there is a growing demand for higher measurement accuracy in the geometric dimensions of oil pump blades. Fig 1. Schematic of blade dimension tolerances For two-dimensional dimensional measurement of workpieces, Shang Xiaoyan et al. [1] calculated the size of parts by fitting the edge points of shaft-type parts with least squares. In order to improve measurement accuracy without increasing hardware costs, sub-pixel edge detection is widely used. Currently used methods mainly include: sub-pixel edge detection based on wavelet transform, interpolation-based sub-pixel edge detection, fitting-based sub-pixel edge detection, and sub-pixel edge detection based on moments. Fu Yu et al. [2] first corrected the distortion of the camera, obtained the camera intrinsic parameters, and then used a sub- pixel edge detection algorithm based on Zernike moments to obtain edges and calculate the size of small connectors. Yongqiang Ren [3] proposed a method for measuring the inner and outer diameters of cylinder liners based on RANSNC (random sample consensus) fitting of sub-pixel contours. However, traditional least squares fitting and global camera calibration cannot meet the accuracy requirements for blade size detection in this paper. To address the shortcomings of traditional camera calibration and least squares fitting of workpiece edges, this project, based on camera partitioning correction, studied a method to improve the measurement accuracy of blade size using an auxiliary measurement mechanism. By assisting the measurement mechanism to determine the edge contour of the blade and fitting the straight line of the blade edge, higher 85 precision sub-pixel edges and fitted lines are obtained, thereby making the blade size more accurate. 2. Principle of Measurement System The project overall first involves image acquisition, followed by image processing of the captured images, and fitting of the acquired points to obtain the blade edge. The measurement principle and system schematic are illustrated in Figure 2 and Figure 3, respectively. Contact between auxiliary measuring mechanism and blade image acquisition algorithm processing result display Illuminate with a light source Camera capture Imaging system calibration Edge fitting calculate fitted line distance blade dimensions Fig 2. Overall schematic flowchart of measurement principle The image acquisition section begins by illuminating the measured blades with a light source, followed by auxiliary measurements assisted by a measuring mechanism. Subsequently, the camera captures the required images. The algorithm processing involves calibrating the imaging system and fitting the edges of the blades. Finally, the dimension display results are obtained. Camera ComputerLens Loading platform Measured workpiece Auxiliary measuring mechanism Fig 3. Schematic Diagram of visual dimension measurement system camera, lens, light source, measured workpiece, auxiliary measuring mechanism, and loading platform. It is capable of capturing images of the measured workpiece and measuring its dimensions. 3. Blade Measurement Process 3.1. Test of Grinding Fineness In computer vision and image processing, calibration refers to the process of determining the internal or external parameters of a camera through some method. The purpose of camera calibration is to map pixel coordinates in the image to physical coordinates in the real world, or vice versa. Camera calibration typically involves using a special calibration board or object and capturing images at different positions and orientations. By analyzing these images and combining them with known physical dimensions and geometric relationships, the internal and external parameters of the camera can be calculated. Currently, traditional calibration methods can be divided into three types: linear calibration methods, nonlinear optimization calibration methods, and two-step calibration methods. These methods all globally calibrate the distortion of the camera parameters. However, in high-precision dimension measurements, it is difficult to accurately map individual sub-pixel points to real-world physical coordinates. Therefore, this system adopts a region-based camera calibration approach, mapping each corner point of the calibration board to actual physical coordinates to perform distortion correction. Fig 4. Calibration board Fig 5. Single region of the calibration board Due to the inability to obtain calibration results in real time, this project adopts a method where calibration is performed first, and then the calibration results are written into a file for direct reference. High-precision calibration boards are used during calibration, with dimensions of 100 mm × 100 mm × 3 mm, as shown in Figure 4. The calibration board has a chessboard pattern, with each grid size being 10 μm × 10 μm. The graphic size is 100 μm × 100 μm, with an overall 86 accuracy of ±0.5 μm. The detected corner points are the intersections of the black edges of the calibration board pattern, with the calibration features being the horizontal and vertical lines of the cross grid. The distance between lines is 100 μm, and each line has a width of 10 μm. To reduce error accumulation, the calibration board is divided into 16 different regions for calibration correction, with a single region of the calibration board shown in Figure 5. During the calibration process, the upper-left corner intersection point of each region is taken as the coordinate origin for that region, and calibration for each region is completed sequentially. The principle is to assume that the calibration board is ideal and error-free, and based on the coordinates of the corner points on the calibration board and their coordinates in the pixel coordinate system, to determine their correspondence. The relationship between the pixel size and the actual size within each region is as follows: Let the CMOS pixel size be p, the lens magnification factor be β, the geometric size corresponding to each pixel size be l, and the geometric size of each two calibration board coordinate intersection points be d. Then, the number of pixels m between each pair of coordinate intersection points can be expressed as: 1 In this system, a telecentric lens with a magnification factor of 0.5x is used. Under ideal conditions, the pixel size of the Hikvision camera used is 3.45 μm × 3.45 μm, and the geometric size of each coordinate intersection point distance is d = 100 μm. Therefore, we can obtain m ≈ 14.493. The calibration algorithm for each individual region proceeds as follows: First, utilize Sobel operators in both horizontal and vertical directions to extract horizontal and vertical lines within the image region. Then, project the extracted horizontal and vertical lines separately, tallying the pixel coordinates and frequencies of the line projections at each location. Calculate the average number of pixels between each line segment in each direction for size calibration. Next, perform image multiplication on the extracted horizontal and vertical lines to obtain the intersections of the two lines in the image, which serve as distorted corner features. Determine the true positions of the corners under ideal conditions based on the physical distance between each line interval on the calibration board and the calibrated size calculated in step (2). Finally, use the corner positions obtained in steps (3) and (4) as reference points to establish a mapping relationship within a 100 mm × 100 mm area. A cubic polynomial fit can be applied to obtain: 2 3 The formula is as follows: (x, y) represents the distorted pixel coordinates; (x', y') represents the real position; a0 ~ a9 are the fitting polynomial parameters in the x-direction; b0 ~b9 are the fitting polynomial parameters in the y-direction. The calibration process is illustrated in Figure 6 Perform the above calibration process sequentially for the 16 segmented regions obtained, obtaining the distortion parameters for each 100 mm × 100 mm area, and write them into an index file. Subsequently, real pixel coordinates can be calculated based on Equation (1) and the index file. 3.2. Blade Edge Extraction The presence of burrs and oil stains caused by the processing technology of the blades results in pseudo-edges during the extraction of the blade edges. Meanwhile, there are rounded transitions around the edges of the blades. To determine the starting and ending positions of the 4 edges of the blade contour, processing is done through the minimum bounding rectangle of the blade image. The original image of the blade is shown in Figure 7. Due to the small defects on the blades and the addition of noise at various stages of the image acquisition process, it affects the area size of the minimum bounding rectangle of the blade as shown in Figure 8. Therefore, it is necessary to perform filtering and sharpening to reduce noise interference. The image after processing is shown in Figure 9. Based on the preprocessed binary image, locate the minimum bounding rectangle of the blade, then use the Canny operator to find the blade edges. Next, compare the edge points with the minimum bounding rectangle and remove those that are beyond or far from the rectangle. If there are no edge points nearby, consider that there is no edge point at that position. Finally, store the edge points on the four edges into four vector lists. The final edge points are shown in Figure 10. After obtaining the coarse edges of the blade, the subpixel edge detection algorithm based on the fitting of the Sigmoid function is utilized to extract the edge subpixel points of the image. This algorithm considers that the edge of the image can be fitted with a Sigmoid curve to obtain the position of subpixel points. Finally, by using the least squares method to fit two relatively straight lines to the subpixel points, one of the fitted lines is shown in Figure 11. 3.3. Dimension Calculation Using Auxiliary Measuring Devices This project's innovation lies in the addition of auxiliary measuring mechanisms, which assist in locating the edges of the blades and fitting straight lines to the edges, thereby improving the measurement accuracy of the blades. In the absence of auxiliary measuring mechanisms, due to the rounded corners around the blades, minor defects on the edges, and various noise factors such as burrs, even with image processing techniques such as filtering, erosion, and dilation, the final edge points around the blades still have relatively large errors compared to the actual ones. As shown in Figure 12, the irregular arrangement of edge points in the upper left corner of the blade after processing is not conducive to the subsequent fitting of edge lines. 87 (a)Vertical edges (b)Horizontal edges (c)Vertical edge projection (d)Horizontal edge projection (e)Vertical edge segmentation (f)Horizontal edge segmentation (g)Calculation of intersection points (h)Ideal and actual Fig 6. Calibration process Fig 7. Original Image Fig 8. Minimum bounding rectangle of the original image 88 (a)Gaussian filtering (b)Erosion and dilation Fig 9. Result of image preprocessing Fig 10. Locally processed edge points Fig 11. Fitted edge line Fig 12. Edge points processed for the upper left corner of the blade Therefore, incorporating auxiliary measuring mechanisms allows for the edges to make contact with the blade at certain distances from its four vertices. Due to the distinctive features of the auxiliary measuring mechanisms (tungsten steel needles), the edge positions of the blade can be better determined. An illustration of the contact is shown in Figure 13, where the two vertices circled in red protrude noticeably, which is unfavorable for subsequent straight line fitting. Therefore, it was decided to fit the edge points in the vicinity of the contact position of the tungsten steel needle, instead of fitting straight lines to the entire edge of the blade. This enhances the accuracy of blade edge positioning and consequently improves blade measurement precision. Fig 13. Contact illustration between the blade and tungsten steel needle Due to the distinctive features of the auxiliary measuring mechanism (tungsten steel needle), when it makes contact with the blade, adjustments are made to the most suitable parameters of the image preprocessing function by observing the characteristic morphological changes of the tungsten steel needle tip and using a calibration plate as the background. This adjustment ensures that the blade image, after preprocessing, yields clearer and more realistic edges. Additionally, because there is a certain distance between the edge contour pixel sets at the positions of the four vertices of the blade, which are unfavorable for edge fitting, when the tungsten steel needle contacts the blade, edge pixel sets in the vicinity of its contact are selected for edge line fitting. This approach enables the fitted lines to be closer to the actual edge positions. Figure 14 illustrates a comparison of edge line fitting with and without the assistance of the tungsten steel needle. In Figure (a), due to the distribution of edge points at the blade vertices, the fitted line has a larger error, clearly inferior to the higher accuracy of the line fitted using edge pixel sets in the local range around the contact position of the tungsten steel needle with the blade, as shown in Figure (b). Fig 14. (a)Without using the tungsten steel needle Fig 14. (b)Using the tungsten steel needle 89 4. Conclusion The experiment involved capturing images of the same blade at seven different positions, resulting in measurements of the blade's length and width, as shown in Table 1. In this table, blades marked with an asterisk (*) indicate those measured using the auxiliary measuring mechanism. Table 1. Blade measurement dimensions (unit: micrometers μm) parameter position 1 position 2 position 3 position 4 position 5 position 6 position 7 repeatability True size Average result Dimensional deviation Blade width 11959.30 11946.09 11945.83 11949.39 11946.86 11948.82 11943.79 4.71 11957.0 11948.58 8.42 Blade length 24955.96 24957.72 24958.53 24946.24 24944.27 24957.12 24953.40 5.34 24948.0 24953.32 -5.32 * Blade width 11955.80 11956.78 11956.07 11955.61 11956.12 11956.13 11956.40 0.35 11957.0 11956.13 0.87 * Blade length 24952.53 24952.63 24952.83 24952.85 24952.65 24952.02 24952.84 0.27 24948.0 24952.76 -4.76 The data in Table 1, it can be observed that the dimensional deviation and the repeatability of the width and length measurements of the blade, obtained with the assistance of a tungsten steel needle, are significantly smaller than the dimensional deviations of the blade measurements without using the tungsten steel needle. Meanwhile, the length dimensional deviations in both cases are not significantly different. Overall, the measurement accuracy is higher when the tungsten steel needle is used to assist in the measurement, which confirms that the auxiliary measuring mechanism can improve the measurement accuracy of the blade dimensions. Acknowledgments This paper was supported by the invaluable assistance and guidance from numerous individuals and organizations. We extend our heartfelt gratitude to our supervisor for their continuous support and invaluable insights throughout the research process. We also express our appreciation to our collaborators and colleagues for the contributions and fruitful discussions. Furthermore, we would like to acknowledge the financial support provided by the school and our advisor, which enabled this research to be conducted. Finally, we are grateful to the technical staff and research facilities for their assistance in our experiments and analysis. References [1] Shang Xiaoyan, Jiang Xu, Wu Jian. Research on Dimensional Measurement Technology of Shaft Parts Based on Image Processing [J]. Tool Engineering, 2012, 46(3): 85-87. [2] Yu Fu, Zhong Shaojun, Xie Min, et al. Research on High- precision Measurement of Small Connectors [J]. Electronic Technology Applications, 2012, 38(9): 144-146. [3] Ren Yongqiang, Tu Dejiang, Han Shu. Dimension Measurement of Diesel Engine Cylinder Liner Based on Machine Vision [J]. Combined Machine Tools and Automated Manufacturing Technology, 2020(9): 151-153.