162 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) ISSN (Print) 2313-4410, ISSN (Online) 2313-4402 © Global Society of Scientific Research and Researchers http://asrjetsjournal.org/ Pointer Temperature and Humidity Meter Detection Based on Machine Vision Lei Genga, Wenke Lib, Jiangtao Xic*, Yanbei Liud a,b,c,dSchool of Electronics and Information Engineering, Tianjin Polytechnic University, Tianjin, 300387, China aEmail: genlei@tjpu.edu.cn, bEmail: toliwenke@gmail.com cEmail: jiangtao@uow.edu.au, dEmail: liuyanbei@tjpu.edu.cn Abstract A new method is proposed to achieve the accurate segmentation and reading of different types of pointer temperature and humidity instruments. The Canny edge detection operator is applied to detect edges of the meter gray image, and the scale line distribution arc and scale lines is obtained according to distribution characteristics of the scale line. The pointer is fitted by concentric method within the radius of the arc. OCR identifies the scale value of the dial, and then determines the corresponding reading number of each scale line and dividing value by mapping relation. Three different types of meters are used to detect, and experiments results demonstrate that the proposed method can achieve accurate segmentation of pointer and scales, as well as accurately calculate the readings of instrument. Key words: temperature and humidity meter; gaussian filter; edge detection; region of interest; canny operator; extracting circular arc; optical character recognition. 1. Instruction The pointer type temperature and humidity meter is widely used in the fields of medicine, agriculture, instrument calibration and the like because of its simple structure, convenient use, low price and high precision. ------------------------------------------------------------------------ * Corresponding author. http://asrjetsjournal.org/ American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 163 At present, the calibration of the temperature and humidity meter is mainly completed by manual reading and manual recording, and is susceptible to the proficiency and mental state of the testing personnel[1]. The use of machine vision and image processing technology to improve the automation of meter verification, reduce the repetitive and simple manual operation, can effectively reduce the cost and time of meter verification and improve verification efficiency [2]. At present, in the pointer-based instrument detection system based on machine vision, it is mainly divided into two parts: pointer segmentation and instrument representation value interpretation. However, due to the overlap of the pointer and the scale value and the complicated dial information, the pointer segmentation deviates from the center line and is different. There is a difference in the type meter graduation value, which also makes the correct interpretation of the instrument representation value greatly increased. For the segmentation of pointers, there are mainly subtraction, Hough transform and least squares [3]. Literature [4] proposes an image subtraction technique with a threshold based on the fact that the pointer is the only element of the instrument image sequence that changes its position. This method requires obtaining the background image in advance, and the image collection environment is strict, and the adaptability and real-time performance are poor. According to the relationship between the pointer and the center of the dial, the literature [5] improves the Hough transform algorithm by double threshold definition and dial center calibration, which reduces the computation time of the algorithm. However, the algorithm needs to be refined before the algorithm is executed. When the refinement produces distortion and pixel redundancy, the pointer extraction deviation is large. In [6], the RANSAC least squares algorithm can locate pointers in complex instrument information, but the algorithm parameters are more strict, and different instrument pointers are extracted with different parameters. Reference [7] uses RGB three-channel gray value changes and improved probability transformation to locate the car instrument pointer. Different color pointers have large differences in the three-channel gray value, and there is also a problem of poor versatility. The interpretation of the value of the instrument is mainly divided into the distance method and the angle method. For example, the literature [8] uses the polar coordinate change method to convert the meter scale line distributed on the ring into the horizontal direction and then according to the relative distance reading. Reference [9] calculates the value of the instrument based on the relative angular difference between the pointer and the nearby scale line. The above two methods need to determine the graduation value of the instrument in advance, but in the actual detection, the graduation value of different instruments is uncertain and cannot meet the requirements of real-time detection. The OCR technology can be used to obtain the scale value of the dial, and the scale line obtained by the division is used to obtain the index value of different meters, which improves the flexibility of the American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 164 interpretation value of the instrument. This paper proposes a temperature and humidity meter detection method based on the scale arc and OCR. Extract the scale arc by calculating the gray scale projection perpendicular to the arc direction; use the OCR technique to read the dial scale value, construct the mapping relationship to obtain the scale line indication value; and generate the equidistant concentric circle within the scale circle arc radius The pointer is fitted, and the meter detection is completed according to the value of the pointer from the pointer to the scale line. 2. Segment meter tick For different types of instruments, the color and distribution direction of the tick marks are different. Based on the arc of the tick marks, the inner and outer edges of the arc are judged to determine whether the tick marks are inside or outside the circle. By setting the appropriate threshold distance, the scale circle is divided by the method of concentric circle subtraction, and the gray value in the ring is statistically determined to obtain the gray threshold to divide the scale line. 2.1 Extract tick arc The exact positioning of the tick arc is the basis for the subsequent segmentation of the tick. Firstly, the ROI is determined by artificial interaction, the image in the ROI is grayed out to obtain a single-channel grayscale image, and Gaussian filtering is used to smooth the image to remove noise in the image. The Canny edge detection operator is used to extract the edge of the contour, and finally the scale arc is accurately obtained by judging the edge distribution. The specific steps are shown in Figure 1. C olor p ictu re ROI G au ssian filterin g C an n y E dge detection E xtraction arc G rayscale Figure 1: Extract arc processing American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 165 2.1.1 Segment image Temperature and humidity meters will be selected at specific temperature points for testing. The temperature is 15, 25 and 35, and the humidity is 30%, 60% and 90%. When testing the temperature and humidity of a certain point, ROI (region of interest) is selected by man-machine interaction. Processing in ROI can not only ensure the temperature value to be detected in the region, but also reduce the processing time of algorithm and improve the accuracy of image processing. 2.1.2 Gaussian filtering In the process of acquisition, the image will be affected by the external environment, resulting in the initial image superimposed noise and other interference signals. These noises will have an impact on image processing in the later stage, and will not be conducive to the extraction of useful information. Through analysis, the image is mainly affected by Gauss noise. Gaussian filtering is used to smooth the pixels in the image domain. The pixels in different positions in the domain are given different weights. The image can be smoothed while retaining the overall gray distribution characteristics of the pixels. The image of three different types of instruments obtained after Gauss filtering is shown in Figure 2. The parameters of Gauss template can be calculated by the Gauss function. The formula is as follows: 2 2 22 2 1( , ) 2 x y G x y e π + − = s s (1) Figure 2: Smooth image 2.1.3 Extract arc American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 166 After Canny edge detection, the edge information of all gray level mutations in the image is obtained. The edge has two shapes: straight line and curve. Because the arc of the calibration line is the longer edge of the curve, the arc of the calibration line is extracted from the edge of the curve. The specific algorithm flow is shown in Figure 3. S tart Im age edge E xtract cu rve ed ge N u m b er of edges of cu rves N>=3 S elect th e lon gest 3 ed ges Yes No Traverse，gray- scale p rojection E n d Figure 3: Extract arc Firstly, the edge of the curve is extracted from the edge information of the image, and the number of edges of the curve is counted to set the set of edge search to improve the efficiency of the algorithm. Finally, traverses all edges in the search set, and chooses the scale arc from different gray projection. The projection sketch is shown in Figure 4, along the edge of the curve at a distance of 1 pixel each, adding 30 gray values in the direction of the dotted line perpendicular to the edge to obtain a gray value projection array. The projection array of the scale line arcs will have an extreme value in the position of the calibration line, and then determine the arc of the scale line. The extraction process is shown in Figure 5. 2.2 Segment scale line According to the characteristic that the scale lines are distributed on the arc, the distribution direction of the scale lines is determined by searching for the straight edges perpendicular to the arc inside and outside the arc, that is, to the inside or to the outside of the arc. After obtaining the distribution direction of the calibration line, the calibration line region is segmented by addingα pixel width inward or outward on the basis of the arc American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 167 radius. The length distribution of the calibration line fluctuates little, and the scale area can be completely divided when theα value is adjusted to 100. In the calibration line area, the foreground has only the calibration line, and the calibration line can be extracted easily and quickly by threshold segmentation. Considering the fact that the gray value of the scale line is bright or dark and the scale line is bright, the points whose gray value of the pixel in the scale line area is greater than the mean value are selected, and the points whose gray value is less than the mean value are selected. The dividing line is shown in Figure 6. Figure 4: Gray projection 1 2 3 4 Figure 5: Extraction process American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 168 Figure 6: scale line 3. Segment pointer The accurate extraction of temperature and humidity instrument pointer is very important for the accuracy of instrument reading. The common methods of pointer extraction include color feature and subtraction[11]. The color feature method is suitable for the case that the color of the pointer is fixed and the discrimination between the pointer and the background is large. It can not complete the unified pointer extraction for the temperature and humidity meters with different color pointers and complicated dial[12]. Subtraction method needs to get the background image beforehand. When there are elements in the background with the same color as the pointer and overlap with the pointer, the overlap part will be filtered out as the background, so the pointer can not be extracted accurately. In order to accurately extract the pointers of different temperature and humidity meters, firstly, the pointer region is obtained in the ROI of the pointer by using the gray threshold segmentation method[13], and then the pointer region is accurately located according to the length and difference of the intersection part of the concentric circle and the pointer region. The specific algorithm steps are as follows: (1) with the radius of the circle as the outer radius and the 1/2 length as the inner radius, the pointer ROI is segmented. In ROI, the threshold is connected to the domain, and the longest connected domain along the radius is selected to locate the pointer initially. (2) Taking the inner radius as the starting radius and the outer radius as the ending radius, the concentric circle is generated every 5 pixels to obtain the intersection area, and the intersection length of each concentric circle and the pointer area is calculated. (3) Areas less than 25 pixels in length are reserved, and the difference in length between adjacent intersecting regions is calculated, and the intersecting region with a difference of less than 3 pixels is American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 169 reserved. (4) Save the centroid of the area obtained in step (3) and fit the line where the pointer is located. 1 2 3 4 Figure 7: Segment pointer 4. Identify the dial scale value The accurate segmentation of the scale values on the dial is a prerequisite for the correct identification of the OCR. According to the characteristics that the scale value on the temperature and humidity meter has zero, the roundness of the connected domain is first compared, and the roundness is calculated by the formula (2), and the connected domain with the roundness greater than 0.5 is selected. Secondly, from these connected domains, a connected domain with similar areas and approximately equal distances from the scale arc is selected, that is, a zero connected domain to be searched. Since the scale value is covered by the pointer, the searched zero connected domain is traversed, and the distance from the centroid to the pointer is calculated to determine whether the scale value is covered by the pointer. The zero connected domain far away from the pointer is left to achieve accurate segmentation of the digital region. 2( ) FC L = ∗π (2) Where F is the connected domain area and L is the longest distance from the center of the connected domain to all contour points. The digital area is distributed to the left of the zero area, and the digital area can be found by panning the appropriate pixel width in the direction perpendicular to zero. 11M , 02M , 20M is the second American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 170 moment of the connected domain. Then the zero region directionω is calculated by the following formula. 11 02 200.5 2(2 , )atan M M Mω = − − (3) Taking the center ( , )C Row Col of the zero region as the starting point, take the circumscribed rectangle width W as the translation distance and calculate the position ( , )P m n of the digital zone. When 0 / 2ω π< < , * ( ) * ( ) m Row W cos n Col W sin ω ω = −  = − (4) When / 2ω π= , m Row n Col W =  = − (5) When 0ω < , * ( ) * ( ) m Row W cos n Col W sin ω ω = +  = + (6) The segmentation results of different pointer occlusion cases are shown in Figure 8. The segmentation results show that the method of finding zero region can accurately segment the scale values which are not covered by the pointer, and avoid the scale value segmentation failure and segmentation error caused by the pointer occlusion. Figure 8: Scale value American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 171 5. Meter reading In order to determine the indication value of the scale line, we need to map the readings obtained by OCR to its corresponding scale line. It is found that the area of the scale line marked with scale value is larger than that of the scale line nearby by statistical analysis of a large number of instruments. The scale line is named as the large scale line and the small scale line without scale value. The digital region of the known scale value is close to the large scale line. To ensure that the scale value is accurately mapped to the large scale line, the specific implementation method is as follows. The corresponding diagram is shown in Figure 9. (1) Traversing all the scale lines to preserve the distance between each scale line and the scale area. (2) Select the three calibration lines with the smallest distance. (3) Calculate the area of the three selected scale lines. (4) Choose the largest scale line to find the large scale line. Figure 9: large scale line. The integer part of the current meter indication is determined by the number m of calibration lines between the line where the pointer is located and the intersection P of the circle and the center Q of the nearest large calibration line. The schematic is shown in Figure 10. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 172 Figure 10: Reading schematic There are three kinds of positional relationships between point P and point Q, if 1 2λ λ> , 1s = − , if 1 2λ λ< , 1s = , if 1 2λ λ= , 0s = . Obtain the scale value V of the point Q according to the mapping relationship between the large scale line and the scale value. D is the division value, so the formula for calculating the integer part of the value is as follows: ( )I D m s V= × × + (7) The decimal part of the value is determined by the distance from the pointer to the nearest scale line. Considering the non-uniform distribution of calibration line of instrument humidity, the density of calibration line is higher in high humidity area and lower in low humidity area. Select two adjacent calibration lines nearest the pointer to calculate the temperature and humidity values represented by the unit distance. Assuming that the distance between the center of mass of two adjacent calibration lines is K and the distance between the straight line where the pointer is located and the M calibration line is σ , the calculation of the indicator value satisfies the following formula: ( / )T I D K sσ= + × × (8) 6. Experimental results and analysis In order to verify the universality and accuracy of the recognition method, three kinds of instruments are selected with different characteristics. The first indicator is black, the scale line is outside the arc, the second indicator line is inside the arc, the third indicator is white, and the scale line is outside the arc. The result of instrument segmentation is shown in Figure 11. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 173 Figure 11: Segmentation result Under the same light condition, the indication values of the above three instruments at the temperature and humidity monitoring points are read. As shown in Table 1 and Table 2, this algorithm can realize the accurate interpretation of the pointer temperature and humidity meter, and the average error between the measured value and the actual value is small, which meets the requirements of temperature and humidity meter detection. Table 1: Temperature test results under different instruments Instrument type Verification point /℃ Measured value /℃ Error /℃ Instrument 1 15.00 15.00 0.00 25.00 24.95 0.05 35.00 35.01 0.01 Instrument 2 15.00 14.96 0.04 25.00 25.03 0.03 35.00 35.00 0.00 Instrument 3 15.00 15.05 0.05 25.00 25.00 0.00 35.00 34.88 0.12 Average — — 0.03 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 174 Table 2: Humidity test results under different instruments Instrument type Verification point /% Measured value /% Error /% Instrument 1 30.00 30.05 0.05 60.00 60.01 0.01 90.00 90.03 0.03 Instrument 2 30.00 29.98 0.02 60.00 60.00 0.00 90.00 89.97 0.03 Instrument 3 30.00 30.04 0.04 60.00 60.01 0.01 90.00 90.03 0.03 Average — — 0.02 8. Conclusion According to the characteristics of the temperature and humidity meter and the artificial interpretation method, respectively locate the circle of the instrument tick mark, divide the tick line, and determine the scale value corresponding to each tick line. finally, the measured value of the instrument is calculated by the relative distance between the straight line of the pointer and the tick mark. The experiment shows that this method is better than the method of representing the value based on the angle model, and the effective information of the instrument can be used more effectively, which is more robust and less affected by the image quality, and it has universality for many kinds of temperature and humidity meter detection. The next step will continue to improve the recognition accuracy and consider the breakage of the tick marks or the detection in case of occlusion. References [1] Chen Shu, Yang Tian. “A pointer meter detection based on improved ZS refinement algorithm”. Computer Engineering, vol. 43, pp. 216-221, 2017. [2] Tang Liang, He Wen, Li Qian, et al. “The algorithm for the recognition of pointer type meter based on space transformation”. Electrical Measurement & Instrumentation, vol. 55, pp. 116-121, 2018. [3] Alegria F C, Serra A C. “Automatic calibration of analog and digital measuring instruments using American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2019) Volume 52, No 1, pp 162-175 175 computer vision”. IEEE Transaction on Instrumentation and Measurement, vol. 49, pp. 94-99, 2000. [4] Wang Bo, Qin Lingsong. “Index meter examination system based on computer vision”. Computer Engineering, vol. 31, pp. 19-21, 2005. [5] Chen Shu, Wang Lei. “Method for testing of pointer instrument based on machine vision”. Computer & Digital Engineering, vol. 44, pp. 1821-1826.2016. [6] Sun Haoyan. Research on recognition system of pointer meter reading based on machine vision, Changchun : Jilin University, 2015. [7] Xun Yang, Zhang Qingrong. “Research on image processing based automotive pointer instruments detection”. Computer Application and Software, vol. 31, pp. 219-221, 2014. [8] Si Jian, Zhang Dong, He Jianguo, et al. “Design of remote meter reading method for pointer type chemical instruments”. Process Automation Instrumention, vol. 35, pp. 77-79, 2014. [9] Liu Di, Bi Duyan, Li Quanhe, et al. “Automatic interpretation algorithm for pointer instruments based on non-uniform illumination” Computer Application and Software, vol.30, pp. 47-48, 2013. [10] John Canny. “Computational approach to edge detection” IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(1): 679-698.vol. 8, pp. 679-698, 1986. [11] Wang Rui, Li Qi, Fang Yanjun. “An automatic reading method of pointer instruments image based on improved angle method”. Electrical Measurement & Instrumentation, vol. 50, pp. 115-118, 2013. [12] Zhang Yongqiang, Di Jinhong, Ma Pengge. “Auto dashboard pointer detection based on machine vision”. Computer Measurement & Control, vol. 23, pp. 1922-1924, 2015. [13] Tao Bingjie, Han Jiale, Li En. “An applied method for reading recognition of index instrument”. Opto-Electronic Engineering, vol. 38, pp. 145-150, 2011.