Начиная с начала 2000 года осуществляется внедрение GHIS в здравоохранении, в рамках принятого проекта о реформирование информ Mathematical Problems of Computer Science 44, 51--58, 2015. Structure-Based Technique for Object Detecting in UAV Imagery David G. Asatryan Institute for Informatics and Automation Problems of NAS RA e-mail: dasat@ipia.sci.am Abstract In this paper, the problem of automated detection of anomalies in the video, filmed in the controlled area on board of the UAV is considered. The complexity of the problem associated with the effect of multiple factors on the quality of the image is pointed out. Expediency of processing methods using the structural properties of the image, stable with respect to a number of noise and distortion is justified. We propose two approaches to use the structural properties of an image. The first approach is based on full hierarchical segmentation and simplification of the image and usage of the appropriate parameters of segments. The efficiency of the approach applied to the problem of detection of smoke and fire is shown. The second approach is based on modeling and analysis of the image gradient field. Then a method of detection of anomalies in the image is proposed. A few examples illustrating the operation of the proposed procedures are considered and effectiveness of approaches is shown. Keywords: UAV, Anomaly search, Image structural properties, Total segmentation, the gradient field, Proximity measure. 1. Introduction The problem of detecting anomalies arises in many areas of science and technology. For example, anomalies, tracked from space, could be oil spills on the sea surface, major forest fires, man-made pollution of the atmosphere, hidden plantations of narcotic plants among other crops, and others. In order to detect such objects unmanned aerial vehicles (UAV) are also widely used, which are often almost the only means used for search and detection of objects of interest. 51 Structure-Based Technique for Object Detecting in UAV Imagery 52 In this paper, we consider the problem of automated detection of anomalies in the video, filmed on UAV in the controlled areas. The anomaly is understood in a broad sense, including relative to objects of both known and unknown images. As it is known, the video captured on board UAVs simultaneously undergoes the variety of effects and distortions hampering efficient processing of information. The main distortion of an image is occurred due to the rapid changes of UAV flight trajectory, resulting in an image of the same object to be different in the different frames. In addition, images of the same objects in the adjacent frames of a video sequence differ both in size and perspective, and the images themselves are fluctuating due to the vibration of UAV board and other factors. Therefore, for the efficient processing of information received UAV, you must either create special approaches and methods or improve the existing ones. At the same time it should be noted that the effectiveness of digital image analysis depends largely on the available prior information about the properties of the image characterizing features of a scene and numerical values of its major parameters. On the other hand, since the analysis results ultimately are estimated by a person, the used methods should take into account the properties and capabilities of human visual system (HVS). In recent years, particular attention is paid to methods of analysis using structural properties and image features that are consistent to HVS decisions. Under the structure of the image is commonly understood the set of existing standard elements in it and the links between them that allow making a judgment on the content of the considered scene. Herewith, as structural (morphological) elements often the image homogeneous area, as well as the edges of homogeneous regions are considered. To find the structural elements they use a number of morphological operations to perform various image transformations [1]. There is also a huge variety of methods based on the use of different combinations of morphological operations, allowing analyzing the scene specifically for a particular purpose. It follows that, for the efficient processing of video, filmed on board the UAV, it is advisable to apply the methods of image analysis which provide results that are independent or weakly dependent on the above type of interference. Therefore, we restrict ourselves to those situations in which when exposed to this type of interference the structure of the image remains fairly stable. In this paper, we describe two methods of analysis based on using structural properties of an image, allowing detecting anomalies on images obtained from video filming by UAV. Results of application of both methods to real data are given. 2. Two Approaches to Analyzing the Structure of an Image In this paper, we follow the statement that image structure which is perceived by HVS is basically identified via mutual dislocations of available segments and edges. Recently, guided by this statement, we have proposed two approaches to the analysis of the image structure. A. Total Segmentation and Simplification of Image. The algorithm of total segmentation and description of the corresponding program system is done in [2]. Let’s describe briefly the algorithm. D. Asatryan 53 Let I I( m,n )= be the pixels intensity matrix of a Grayscale image, m , ,..., N ;= −0 1 1 n , ,...,M= −0 1 1, where { }I( m,n ) , ,...,∈ 0 1 255 . Let the numbers { }j , ,...,θ ∈ 1 2 254 , j , ,...,L= +1 2 1 , { }L , ,...,∈ 1 2 254 satisfy the inequalities L...θ θ θ< < <1 2 . Then the interval of intensities is divided into L +1 subintervals LT , T ,..., T +1 2 1 . Call a set of pixels cS of the image I connected, if any pixel of that set has at least a “neighbor” from the same set. Call a connected set of pixels cS the segment, if all pixels of that set have intensity belonging to the same subinterval from LT , T ,..., T +1 2 1 . Let a segmentation algorithm ξ be chosen, and we have all segments KSSS ,...,, 21 of an image I as a segmentation result. This means that а) K i i S S = = 1  , i , ,...,K ,= 1 2 б) i jS S∩ = 0 , if i j≠ , i, j , ,...,K= 1 2 . Then we say that the total segmentation is performed. Total hierarchical segmentation means the sequential application of segmentation procedure to the given image at maxL , ,..., L= 1 2 , where maxL ≤ 254 . The procedure of substitution of intensities of all pixels of each segment for its average value is called an image simplification. The described procedure of total segmentation can easily be generalized to the case of color image. For that it is necessary to decompose the color image into components of the used color space, perform total segmentation of each color component and then the derived components convert to color image. The values of parameter L can differ for different components. Thus, the result of total segmentation characterizes the image structure which is varying weakly at different values of parameter L . In other words, the number, dislocation and relative sizes of segments remain almost the same. This fact explains the HVS property to recognize the scene content at different sizes and orientations of an image. Described technique was applied to the problems of recovery of damaged images [3] as well as in smoke or fire detection procedures using UAV imagery [4]. 2.2. Modeling and analyzing the gradient field of an image. The basis for usage of gradient field is the well known fact that HVS confidently recognizes the content of an image by the aggregate of available edges. Methods of edge detection in an image are basically based on gradient field analysis using different mathematical methods [1]. One of the approaches to gradient field processing is its presentation as a two-dimensional random variable and creating certain statistical models for image structure investigation [5]. Let H HG G (m,n)= and V VG G (m,n)= be the horizontal and vertical gradients of an image I correspondingly, and M M(m,n)= be the gradient magnitude, where n),(mG+n),(mG=n),M(m, 2V 2 H . (1) a. Model for dominant orientation of an image is based on the usage of the term of orthogonal regression [6], the equation of which is as follows: ( ) ( )( ) ( )H H HV H H V V V V HV H H V V g g g g C µ ρ µ µ µ ρ σ σ σ σ  − − − − − + =  −    2 2 2 2 2 2 21 1 , (2) Structure-Based Technique for Object Detecting in UAV Imagery 54 where Hµ , Vµ , Hσ , Vσ mathematical expectation and MSE of random variables HG and VG , HVρ is the coefficient of correlation between them, C – is a constant. Dominant orientation α of an image is determined as follows: ( ) HV H V H V HV * tg ρ α σ σ σ σ ρ = − + − + 22 2 2 2 2 2 4 . (3) Models (1)-(3) were applied in different tasks for analyzing of images with different sizes and orientations [6, 7]. b. Model for distribution of magnitude gradient. Let’s assume that the gradient magnitude (1) is a random variable with Weibull distribution density ,0,exp),;( 1 ≥               −      = − x xx xf ηη σσσ η ση (4) where 0>η - shape parameter, 0>σ - scale parameter. It must be noted that the fact of using only two parameters in this model has a significant advantage in certain tasks of image processing, connected with searching in big data bases [8, 9]. Particularly, images similarity assessment measure is proposed in [5], which shows the proximity of images having gradient magnitudes with Weibull distribution densities f ( x; , )η σ1 1 1 and ),;( 222 σηxf . The measure is determined by the formula as follows: ),max(),max( ),min(),min( 2121 21212 σσηη σσηη =W , 10 2 ≤< W , (5) where the gradients are calculated using Sobel operator and the parameters in (5) are estimated by the method of moments. 3. Technique for anomaly detection on an image. Let us have an image of a scene on which the images of certain objects of unknown form, sizes and orientation can be there. A feature by which the object can be characterized is its difference from the surrounding space by structural properties. It is required to detect an area which contains the object and select it as clearly as possible. Such statement of this problem has many uncertainties, but very often even an approximate solution of the specified task has a practical significance because the reliability of solution can be increased by multiple observations from different positions and trajectories of UAV. The goal of the present investigation is to show that the information about structural properties of the image helps to solve the problem of anomaly detection by UAV imagery. The algorithm of object searching includes the following steps. Step 1. Select on a scene a part of image which wittingly doesn’t contain any anomaly. The size of the part must exceed the size of the searching object a little. Step 2. Using sliding window consequently estimate the proximity measure W 2 between the chosen part (at Step 1) and the current slide. Step 3. Select the current slide if W 2 exceeds the threshold given beforehand. The result of performance of these steps will differ from the surrounding space by structure. It can be noted that probabilities of true detecting and false alarm errors at the described procedure application significantly depends on signal/noise ratio and can exceed the admissible values. Therefore, the detected anomalies must be investigated additionally by another method. D. Asatryan 55 3. Results of Experiments Some experiments were performed to detect anomalies of different types. Results of two experiments are given below. Experiment 1. Let's suppose that a tank or another big machine is hidden in a forest, and we want to detect an anomalous part by UAV imagery. In Fig. 1a an image is given of size 307x214 from video frame which presumably contains an object. Estimating the situation visually we can note that the image of tank and its nearest surroundings differ from the remaining part of the whole image. In Fig. 1b (slightly increased for demonstrability) an image is shown of size 47x45 from other frame of video which doesn’t wittingly contain an anomaly. Performing the above described procedure we have detected an anomaly, i.e., the part of the whole image with the lowest similarity with the chosen image of Fig. 1b. In Fig. 1c the binarized image of similarity map is shown, calculated using similarity measure W 2 , which allows to illustrate the corresponding part of the whole image with an anomaly object (i.e., hidden tank). Fig. 1. Results of searching for detecting an anomaly. a- video frame with hidden object, b – image part without anomaly, c – binarized map of proximity measure, d – the part of whole image with minimal values of proximity measure. Experiment 2. Searching of object of interest using Google Map web mapping service. Here we solve the following problem. Suppose that formerly someone took a picture of an urban quarter using UAV (Fig. 2a). Later he wanted to find that territory using Google Map web mapping service for determination of more detailed information on that territory (more precise coordinates, dislocation of objects, etc.). Suppose that the chosen part of Google Map (Fig. 2b) contains the object of interest, but distance, orientation, perspective and other information regarding the picture of Fig. 2a are unknown. The goal of this experiment is to find the object of interest in whole image of Fig. 2b (i.e., a part which corresponds to the image of Fig. 2a). a b c d https://en.wikipedia.org/wiki/Web_mapping https://en.wikipedia.org/wiki/Web_mapping https://en.wikipedia.org/wiki/Web_mapping Structure-Based Technique for Object Detecting in UAV Imagery 56 Using the above described searching procedure with sliding window (by analogy with Experiment 1) the part of map can be found which is maximally similar to the image of Fig. 2a (see rectangular frame in Fig. 2c). In Fig. 2d the increased image of that frame is shown. It can be seen that the frame contains the necessary information though the scale and orientation differ from the object of interest. For further specification of scene the described procedure can be repeated for another part of map, which contains the object of interest more precisely. Fig. 2. Results of searching the object of interest on a map. a – chosen area of interest. b – part of Google map presumably containing the object of interest. c – found part of map with maximal values of proximity measure. d – increased image of found part. a b c d D. Asatryan 57 4. Conclusions In this paper, a problem of automated detection of anomalies in the video, filmed in the controlled area on board of the UAV is considered. Due to complexity of the problem associated with the influence of multiple factors on the quality of UAV imagery it is reasonable to use the structural properties of images which are stable with respect to a number of noise and distortion. The proposed approaches and procedures allow solving effectively the problem of detection of anomalies in images filmed by UAV. The considered examples of applications show the effectiveness of the proposed approaches and procedures. References [1] R. C.Gonzales and R.E. Woods, Digital Image Processing. 2-nd Edition, Prentice Hall, 2002. [2] D. G. Asatryan, G.S. Sazhumyan and H. S. Shahverdyan, “Technique for coherent segmentation of image and applications”, Mathematical Problems of Computer Science, IIAP, Yerevan, Armenia, vol. 28, pp. 88-93, 2007. [3] [Online]. Available: http://www.rau.am/cct/ [4] D. Asatryan and S. Hovsepyan, “Vision based technique for smoke and fire detection in monitored forest terrain”, In Proceedings of CSIT 2015, Yerevan, pp. 129-132, 2015. [5] D. Asatryan and K. Egiazarian. “Quality assessment measure based on image structural properties”, Proc. of the International Workshop on Local and Non-Local Approximation in Image Processing, Finland, Helsinki, pp. 70-73, 2009. [6] D. Asatryan, K. Egiazarian and V. Kurkchiyan, “Orientation estimation with applications to image analysis and registration”, International Journal “Information Theories and Applications”, vol. 17, no. 4, pp. 303-311, 2010. [7] D. Asatryan, V. Kurkchiyan and M. Bagramyan. “A method for quality assessment of image resizing algorithms”, Mathematical Problems of Computer Science, vol. 36, pp. 128-132, 2012. [8] Д. Г. Асатрян, В. В. Куркчиян и Л. Р. Харатян, “Метод классификации текстур с использованием структурных характеристик изображения”, Компьютерная оптика, N3, с. 574-579, 2014. [9] D. Asatryan and M. Zakaryan, “Novel approach to content-based video indexing and retrieval by using a measure of structural similarity of frames”, International Journal "Information Content and Processing", vol. 2, no. 1, pp. 71-81, 2015. Submitted 04.08.2015, accepted 25.11.2015 Structure-Based Technique for Object Detecting in UAV Imagery 58 ԱԹՍ տեսանկարահանված պատկերի կառուցվածքի հիման վրա օբյեկտի հայտնաբերման մեթոդ Դ. Ասատրյան Ամփոփում Անօդաչու թռչող սարքերը (ԱԹՍ) լայնորեն կիրառվում են տնտեսական, բնապահպանական, ռազմական և այլ բնագավառներում: Սույն աշխատանքը նվիրված է հսկվող տարածքում նկարահանման միջոցով օբյեկտներ հայտնաբերելու խնդրին, հենվելով պատկերի կառուցվածքային հատկությունների օգտագործման մեթոդաբանության վրա: Առաջարկվել է պատկերի կառուցվածքը բնորոշող երկու մոդել, որոնցից մեկը կապված է պատկերի լրիվ հատվածավորման և պարզեցման արդյունքների օգտագործման, իսկ մյուսը՝ պատկերում առկա եզրագծերի և սահմանների բազմության վիճակագրական վերլուծության հետ: Առաջարկվել են պատկերում ընդհանուր ֆոնից օբյեկտի՝ որպես անոմալ տիրույթի, առանձնացման ընթացակարգեր: Մասնավորապես, դիտարկվել են ծխի և կրակի, ինչպես նաև Google map համակարգից ստացված քարտեզների վրա նախօրոք գրանցված օբյեկտի հայտնաբերման օրինակներ: Իրական տվյալների վրա ցույց է տրվել առաջարկված մոդելների և համապատասխան ավտոմատացված համակարգի արդյունա- վետությունը: Метод обнаружения объекта по видеосъемке БЛА, основанный на использовании структуры изображения Д. Асатрян Аннотация Беспилотные летающие аппараты (БЛА) широко используются в народном хозяйстве, в задачах защиты природы, в военном деле и других областях. Статья посвящена задаче обнаружения объектов по видеосъемкам БЛА на контролируемой местности, основываясь на методологии использования структурных свойств изображения. Предложены две модели, характеризующие структуру изображения, одна из которых связана с использованием результатов полной сегментации и упрощения изображения, а другая основана на статистическом анализе множества имеющихся в изображении краев и границ. Предложена процедура обнаружения объекта на общем фоне – как аномалии в изображении. В частности, рассмотрены примеры обнаружения дыма и огня, а также обнаружения предварительно регистрированного объекта по карте, полученной при помощи системы Google map. На реальных данных показана эффективность предложенных моделей и автоматизированной системы обнаружения объектов.