Microsoft Word - cet-01.docx CHEMICAL ENGINEERING TRANSACTIONS VOL. 46, 2015 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors: Peiyu Ren, Yancang Li, Huiping Song Copyright © 2015, AIDIC Servizi S.r.l., ISBN 978-88-95608-37-2; ISSN 2283-9216 Estimation of Depth Information of Image Based on Energy Distribution Jianyi Li*, Quanbo Yuan North China Institute of Aerospace Engineering, Langfang, Hebei, 065000, P.R. China. Ljianyi003@126.com Image segmentation is an effective way to analysis image, which has received more and more attention. The traditional image segmentation technology can't get the result of the semantic image segmentation. Because the image is the projection of the 3D world in the 2D plane, it is necessary to combine the scene depth information to guide the segmentation of the image. In this paper, it analysis the shortcomings of the traditional segmentation technology, and describe the way of the image segmentation through the automatic measurement of the depth information of the digital image, and then complete the image of the auto focus function. 1. Traditional Method in Image Segmentation Image segmentation is a multi-interdisciplinary research direction and design disciplines including artificial intelligence, machine learning, pattern recognition, psychology, visual neurophysiology and cognitive science. In the early stage of image segmentation, the image segmentation algorithm is based on the color appearance attributes, and the spatial properties of pixels are ignored, and the segmentation results are not satisfactory. Several typical image segmentation methods are introduced in this paper. 1.1 Image segmentation technology Early image segmentation is processed by gray level image, such as gray histogram of the threshold segmentation, fuzzy set theory image segmentation, etc. These methods have good effect on the image segmentation results with different background and background gray value. However, the medical images of the foreground and background are not significant. The image segmentation algorithm is divided into the following four categories. 1) Segmentation technology based on threshold How to select the appropriate threshold is the key problem of threshold segmentation. One of the simplest methods is to start from the gray histogram of the image, first get the probability distribution of each gray level, and then select one or more appropriate threshold according to a criterion to determine the ownership of each pixel. 2) Segmentation technology based on edge This method is mainly based on the discontinuity of the gray level of the image, and realizes the segmentation of the image by detecting the boundaries between different regions. Although there are some differences in the accuracy of detection and edge location accuracy, it has a common shortcoming: it can't get the continuous single pixel edge, which is very important for segmentation. So this method needs to be corrected after edge detection. 3) Segmentation technology based on region characteristic There are two forms of the technology: regional growth and separation or merge. The former is from a single pixel, and gradually merged to form the desired segmentation results; the latter is from the whole image, gradually split or merged to form the required segmentation results. Different from the threshold method, this method not only considers the similarity of pixels, but also considers the adjacency of the space, so it can effectively eliminate the interference of the isolated noise, which has strong robustness. Moreover, whether the merger or division, it can be divided into pixel level, so you can guarantee higher segmentation accuracy. DOI: 10.3303/CET1546144 Please cite this article as: Li J.Y., Yuan Q.B., 2015, Estimation of depth information of image based on energy distribution, Chemical Engineering Transactions, 46, 859-864 DOI:10.3303/CET1546144 859 4) Segmentation technology based on statistical pattern classification Patterns can be defined as the quantitative or structural description of the target or other interested parts in the image, and the image segmentation can be regarded as a pattern classification process based on pixels. The process includes two steps: feature extraction and pattern classification. The threshold segmentation is equivalent to the pattern classification in one dimension (gray level) or two-dimensional (co-occurrence matrix) feature space, which is used to reflect the spatial information of pixels and their neighborhood pixels. In order to improve the segmentation effect, we naturally think of using the high dimensional features that can make full use of the image information to describe each pixel. Analysis of the image segmentation techniques based on color feature clustering, they are usually unable to obtain the results of the image segmentation results, so it need to look for new image segmentation techniques. Traditional image segmentation technology to image is deal with as a two-dimensional model. However, the image is the projection of the 3D world in the two-dimensional plane. Therefore, it is necessary to combine the scene depth information to guide the image segmentation. This will provide a possible solution to the image segmentation results with semantic meaning. In this context, the scene depth information is introduced, which effectively solves the problem of over segmentation and less segmentation based on the color appearance of the image segmentation algorithm. 2. Depth Information Estimation of Micro Image Accurate measurement of digital image depth information is widely used in computer vision, automatic control, medical and other fields. Various depth information extraction methods, such as stereo vision, depth of focal depth, motion parallax, etc., are mentioned in the literature. The method of Depth From Defocus has caused wide attention. The focal depth is the focal plane distance of the object Δ z , as shown in Figure 2.1. The method of the focal depth is calculating the depth information of the image zδ based on Δ z . An optical imaging system is shown in figure 2.1. Figure 2.1: An optical imaging system Gauss imaging formula(1) 1 1 1 = + + Δ −f if d z d zδ (1) In equation (1) can be obtained equation(2) − − + + Δ = − − i i f f f i fd f z d d zd fd z d z f δ δ δ (2) Equation (2) reflects the relationship between the distance of the object and the distance of the image. The equation (2) in the denominator is determined by di, while the δz [df/di– f /di] is simplified to [ ]1 1 1− +z M Mδ , then 2Δ ≈ z z M δ (3) Which M is the magnification of the optical system? Equation (3) is the relationship of scattered focal distance between the object and the image. 860 2.1 Depth estimation algorithm The implementation of the algorithm is divided into two parts: 1) the relationship between the depth and the high frequency energy parameters is found by the calibration experiment. And then establish the depth calculation model. 2) real-time measurement of the depth of the micro operation tool. First, the specific implementation process of depth equation is as follows:  A set of fuzzy images of a set of micro tools at different depths is acquired with a set of 2 μm intervals.  The use of high pass filter for this set of sequences of fuzzy image processing.  For each image, the inverse Fu Liye transform is processed, and the high frequency energy parameters are obtained by the effective pixel value of each image.  The relationship between depth and energy parameters is described, and the depth equation is obtained by means of least square polynomial fitting. The expression of the equation is as follows: 2 3 0 1 2 3Δ = + + + + n h h h n hz a a E a E a E a E (4) Where 0 1 2, ,  na a a a is the fitting coefficient, ∆z is for the depth from defocus, Eh is energy parameter. Real time estimation of the depth of the actual micro operation tool is completed in second steps. When the micro operation tool reaches a certain position, the camera can obtain the microscopic image of the current position, find the energy parameter Eh, use the known depth model and calculate the current depth value hD . 2.2 Experiment results This paper is oriented to the computer controlled micro manipulation robot for biomedical engineering, which is used in the imaging of the target object is obtained by the micro manipulation robot glass micro needle. The specific process is as follows:  The mobile micro manipulator arm is moved to the center of the microscope field of view, and then the robot arm is adjusted along the Z axis until the most clearly image is obtained.  Control the robot arm to move along the Z axis in a fixed distance (2μm), until the target (micro needle) image can’t be identified, and this position is used as the end point of the calibration. So we got a series of fuzzy images with different depth.  According to these images, the energy parameters hE of each of the micro needle images (as shown in Figure 2.2) are obtained, which is obtained with a two-tuples < ,h kE x >, which kx corresponds to the depth value of the microscopic image.  A number of two-tuples in step (3) to establish a fitting curve, to get its various coefficient values ka . At this point, the establishment of the mathematical model is completed. 2 32.2129 1.5269 0.4119 0.06.3Δ = − + − +h h hz E E E 4 50.0053 0.0003− + +h hE E ο (5) Figure 2.2: The relationship between the depth and the energy of the micro 861 Figure 2.3: Image of a micro needle under a microscope. 3. Micro Image Depth Information Measurement After the above steps, the mathematical model of the image depth information and energy is established. When a picture is taken for an unknown depth information, the energy of the corresponding region is calculated, and the corresponding depth information can be obtained from the model (5). In order to verify the reliability of the algorithm, we use 100 test images to test the model and compare the results with the actual values. The error of the two values is shown in Figure 3.1. Figure 3.1: Error value of digital image depth information 3.1 Analysis of experiment results 1) Advantages of the algorithm of depth information measurement based on energy distribution The maximum error can be seen in the range of 3 μm. The algorithm has the following advantages:  Real time performance is good, because once the corresponding mathematical model is established, the depth information of the unknown image can be obtained by calculating (such as, the highest exponent of the mathematical model coefficient is 6 in the above experiment). This can be done in a modern machine, only a dozen of the instruction cycle can be completed, the required time should be in milliseconds.  Relatively high accuracy. By Fig 3.1, it is known that the maximum error of the test results is 3 nm. For the operation of the cell under the microscope, it can meet the requirements.  Have certain robustness. In the process of realizing the algorithm, the results of the specific shape of the target object in the micro image are relatively small, because of the statistical energy value of a region. That is, different mathematical models can be obtained by using different shapes of target objects. But what we care about is not the mathematical model, but the depth information of the image. 2) Defects of the depth information measurement algorithm based on energy distribution  Effect of glass on the result. Because the micro needle is made of glass at high temperature drop down. In this way, the microscopic images observed under the microscope will appear hollow phenomenon, such as the same micro needle 3.2., changing the light intensity of the objective lens, the hollow part of the image will be expanded. As a result, there is no doubt that the statistical energy will be changed greatly, which makes the error of the established mathematical model. Eventually lead to error increases. 862 Figure 3.2: The influence of the shape of the micro needle on the mathematical model Figure 3.3: Effect of noise on accuracy  The micro needle attitude have more influence on the results, so that the practical application of the algorithm is limited. In the process of establishing a mathematical model, the attitude of the micro needle is always fixed, the distance of the focal plane of the needle is changed. But in the practical application process, such as the injection of chemical reagents or fixed, moving target cells. During this condition, the position and orientation of the micro pin can be changed, and the energy in the fixed area of the micro pin is bound to be changed, so that the final result is likely to be larger error.  The target energy testing region is related to the shape of the micro needle. In determining the target area, it is often required to combine the shape of the micro needle. The micro needle is because under high temperature conditions, and through the manual drawing, its thickness is often not uniform. Moreover, the needle is often the root of the rougher; the tip of the needle is relatively small. In Fig 3.2, if the target area is too large, it will cause a large impact on the region's energy value. The result will be the energy change of the tip of the needle. If the target area is relatively small, the regional energy change is affected by the noise, which makes the region energy appear jump, as shown in the upper left part of the Fig 3.2. So as to influence the establishment of the mathematical model. In this paper, the two factors, the final selection of the needle tip of the 1/3 part of the target area, received better results, as shown in Fig 3.3. 4. Conclusions In this paper, we use the basic principle of image segmentation, consider the over segmentation and less segmentation of the traditional image segmentation algorithm, introduce the algorithm of the image depth estimation, and give a try to the algorithm of the depth of the micro image, and give an analysis of the advantages and disadvantages of the algorithm. Experiments show that this method can be used to accomplish automatic focusing and depth estimation in the practical micro operating system. Acknowledgements The paper is supported by Scientific & Technical Research Foundation of Hebei Province, China(Grant NO. 15K55403D) and Scientific & Technical Research Foundation of Langfang in Hebei Province, China(Grant NO. 2015011057).This paper is also supported by supported by Scientific & Technical Research Foundation of North China Institute of Aerospace Engineering, China (Grant NO. XJTD-201409). 863 References Aslantas V., 2007, A depth estimation algorithm with a single image [J]. Optics Express, 15(8): 5024-5029. DOI: http://dx.doi.org/10.1364/OE.15.005024 Bianco L., Wilczak J.M., 2008, White A B. Convective Boundary Layer Depth Estimation from Wind Profilers: Statistical comparison between an automated algorithm and expert estimations [J]. Journal of Atmospheric & Oceanic Technology, 25(8): 1397. DOI: http://dx.doi.org/10.1175/2008JTECHA981.1 Cheng H., Wang L.H., 2011, Survey of depth estimation algorithm for three-dimensional video [J]. Journal of Wuhan Institute of Technology, Deng X.L., Ni J.Q., Dai F., et al., 2012, Depth estimation algorithm from monocular image based on LLOM [J]. Application Research of Computers, 29(11): 4357-4359. Fahmy A.A., 2013, Stereo Vision Based Depth Estimation Algorithm In Uncalibrated Rectification [J]. International Journal of Video & Image Processing & Network Secu. Giard J., RondaoAlface P., Gala J., et al., 2009, Fast Surface-Based Travel Depth Estimation Algorithm for Macromolecule Surface Shape Description [J]. Computational Biology & Bioinformatics IEEE/ACM Transactions on, 8(1): 59-68. Huang D., Versnel P.A., 2000, Depth estimation algorithm applied to FTG data [J]. Seg Expanded Abstracts, (1): 2484. DOI: http://dx.doi.org/10.1190/1.1816076 Jeong J.S., Lee D.J., Shin Y.N., et al., 2013, A Relative Depth Estimation Algorithm Using Focus Measure [J]. Journal of Korean Institute of Intelligent Systems, 23. DOI: http://dx.doi.org/10.5391/JKIIS.2013.23.6.527 Jiang Y., Qu Z., Liu W., et al., 2006, A fast wavelet-based depth estimation algorithm for arbitrary stereo configuration[C]// Systems and Control in Aerospace and Astronautics, 2006. ISSCAA 2006.1st International Symposium on IEEE,: 5 pp. -701. Lai S.H., Fu C.W., Chang S.Y., 1992, A Generalized Depth Estimation Algorithm with a Single Image [J]. Publication, 14(4): 405-411. Liu Q., Xiao H., 2012, Semi-Global Depth Estimation Algorithm for Mobile 3-D Video Applications [J]. Tsinghua Science & Technology, 17(2): 128-135. DOI: http://dx.doi.org/10.1109/TST.2012.6180038 Malik A.S., Choi T.S., 2008, A novel algorithm for estimation of depth map using image focus for 3D shape recovery in the presence of noise [J]. Pattern Recognition, 41(7): 2200–2225. DOI: http://dx.doi.org/10.1016/j.patcog.2007.12.014 Ren J., Aggoun A., 2003, Mccormick M. A novel object depth estimation algorithm for integral 3D images [C]// Visual Information Engineering, 2003. VIE 2003. International Conference on IET: 198-201. DOI: http://dx.doi.org/10.1049/cp:20030521 Wang X.Q., Shen L., Jiang Y., et al., 2011, A New Improved Depth Estimation Algorithm Based on Gradient Feature for Preserving Consistency among Views [M]// IEEE: 37-40. Xiu X., Liang J., 2010, An improved depth map estimation algorithm for view synthesis and multiview video coding [J]. Proceedings of SPIE - The International Society for Optical Engineering. DOI: http://dx.doi.org/10.1117/12.863245 Yuan H., Chang Y.L., Liu X.X., et al., 2010, A Depth Estimation Algorithm for Preserving Spatial Accuracy and Temporal Consistency [J]. Journal of Beijing University of Posts\s&\stelecommunications, 33(4): 26-29. 864