Adv Syst Sci Appl 2024; 01:20–28 Published online at https://ijassa.ipu.ru. Methods of Estimation and Processing of Aerospace Information in Image Contouring Problem Andrey A. Galyaev*, Alexander V. Dobrovidov, Pavel V. Lysenko, Leonid M. Berlin, Maria A. Kolygo V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia Abstract: The article proposes approaches to the solution of the segmentation problem of real satellite images based on random Markov vector fields and new statistical textural characteristics for gray-level co-occurrence matrix. The authors introduce a novel image textural characteristic called statistical complexity that successfully distinguishes deterministic structures on naturally or artificially noised images. Software modules have been developed and satellite image examples are processed in order to determine the contours of the land structures and the presence of static and dynamic objects on the ocean surface. A comparison of two segmentation methods on real images has been made, which showed the prospects of their use in automatic image processing. Keywords: Earth’s surface monitoring, image segmentation and contouring problem, aerospace information processing 1. INTRODUCTION Aircraft and space monitoring of the Earth’s surface using optical and radio devices is the main data source for building electronic maps for various purposes. In the problem of constructing electronic terrain maps based on data obtained by aerospace carriers, the task of constructing contours of objects of natural and artificial nature often arises [1, 16, 19, 22]. Similar texture analysis tasks emerge for oil spill detection on the ocean surface [23], as well as for classifying atmospheric phenomena and cloud types [5]. Another problem is associated with type determination of natural and artificial waves on the ocean surface [9,17], as well as finding archaeological or sunken objects [12]. Some segmentation problems include the shoreline of lakes and seas, the border of forests and ice fields, etc. Others include the boundaries of buildings, roads, transmission lines, etc. [21] At the same time, one of the main tasks of creating electronic maps is the task of outlining objects for their further vectorization and compressed representation. The totality of measurements of various physical quantities on the sea surface, deep flat sections and flat sections in the atmosphere generate measurement fields, the structure of which varies depending on various processes occurring both under the sea surface and above the surface [6, 14]. Such digitized data fields are the source material for making decisions about the presence of various anomalies [20] in the observed field implementations and highlighting their shape. However, direct handling of physical models is extremely inconvenient, since they are described, as a rule, by complex nonlinear equations [24]. The procedure of contouring is often reduced to operator delineating with an electronic pencil along the visible boundaries of objects. It is proposed to replace this heavy unproductive work ∗Corresponding author: galaev@ipu.ru PROCESSING OF AEROSPACE INFORMATION IN IMAGE CONTOURING PROBLEM 21 with an automatic contouring procedure, which will delineate most of the visible objects in the image. A small part of the remaining ”difficult” objects can be handled by operator. To solve the problem of image segmentation, the article suggests two approaches: the one based on the use of a spatial interaction model (SPM) and the second based on new textural characteristic called statistical complexity. Such models can describe some piece- wise homogeneous spatial structures, where homogeneous areas are called textures. The main feature of the SPM is that they allow taking into account the intra-textural dependence of observations. This circumstance turns out to be decisive, since, for example, experience with radar images has shown that it is statistical relationships that are the characteristic features by which it is possible to distinguish objects in noisy images. The usage of statistical complexity provides an advantage compared to other image textural characteristics for the detection problem of the deterministic signal component [2], [3]. The application of information models developed in this research and their modification to the characteristics of real images make it possible to obtain stable boundaries of objects and reliably distinguish their contours. The current article describes the results of segmentation algorithms on noisy images based on Markov’s field and novel statistical feature called statistical complexity. 2. INFORMATION MODEL OF SPATIAL INTERACTION To develop an automatic contouring procedure, an effective information approach is proposed, in which a data field is considered as an implementation of a random field, where each point of the field is statistically related to the surrounding points according to statistical laws. A set of field points with the same (or similar statistical characteristics) form a field segment. Therefore, the task of dividing the implementation of the observed field (hereinafter the image) into areas with different statistical characteristics is called the image segmentation task. 2.1. Markov Model Approach The basis of the SPM is a set of conditional distributions of the values of the observed field, defined at each point of the image at the specified values of the field at neighboring points P {xij|xkl, (k, l) ∈ Sij}, (2.1) where Sij is the set of neighboring points around a point (i, j) called a pattern, i.e. Sij = {(k, l) : (k, l) ̸= (i, j)}. Another way to define the field model is given by the coupling equation xij = φ(Sx ij, ξij), (2.2) where φ is a given Borel function, {ξij} is a field of mutually independent random variables, and Sx ij = {xkl : (k, l) ∈ Sij} is a set of observations whose indices belong to the template. When constructing algorithms for statistical image analysis, linear auto-regressive models of two-dimensional fields are most often used xij = µ+ ∑ (k,l)∈Sij akl(xi−k,j−l − µ) + bξij, (2.3) where µ is a mean value, akl and b are some constant model values. Depending on the type of template, there are causal, semi-causal and non-causal models. It turns out that only causal models allow us to obtain convenient recurrent image segmentation procedures consistent with a certain (for example, scanning) method of obtaining image data. It should be noted that models (2.2) and (2.3) are also used to generate random fields with Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) 22 A.A. GALYAEV, A.V. DOBROVIDOV, P.V. LYSENKO, L.M. BERLIN, M.A. KOLYGO specified statistical properties in model experiments, and in this case only the causal model allows generating the next field value from those already calculated up to this point. Another class of commonly used models includes Markov models that generate fields with a finite number of states θij . For these models , textures are described directly by a set of conditional probabilities P {θij|θkl, (k, l) ∈ Sij}. The mottled structure of the image (for example, a sea surface) containing anomalous areas, can be described as a composition of various textures generated by the SPM. The approach to texture composition proposed here allows us to achieve the following important results: 1. take into account the relationship of all observations in the image field; 2. make the border between homogeneous textures blurred, which directly corresponds to the actual data obtained; 3. get a spotty structure of arbitrary random shape; 4. to build an informational mathematical model, which is a constructive basis for the synthesis of recurrent image segmentation procedures. A texture composition with the specified properties is constructed as follows. First, a Markov field {θij} is generated based on a given system of conditional probabilities. The composition of textures is obtained using the equation xij = µ(θij) + ∑ (k,l)∈Sij akl(θij)(xi−k,j−l − µ(θij)) + b(θij)ξij, (2.4) in which the coefficients are controlled by a Markov field {θij}. Now this image is the source for its segmentation into areas with the same statistical properties. Model (2.4) allows one to use optimal recurrent algorithms developed by the authors for segmentation, while making a minimal error. However, to do this, it is necessary to use all the statistical characteristics of both the Markov field and the coefficients of equation (2.4). However, in practice, it is impossible to obtain the characteristics of an unobservable Markov field. Therefore, a modified cumulative sum algorithm is proposed here for segmentation, using various sample statistics obtained directly from the noise image of the type. Therefore, the linear structure of the solution is caused by a row-by-row method of calculating statistics. There are several ways to get rid of this. In this paper, a nonlinear smoothing filter with windows of various sizes is proposed. In addition to the likelihood ratio statistics, one can use simpler statistics in the form of a square of the second difference: ∆ij = (xij + xi−1j−1 − xi−1j − xij−1). (2.5) Studies have shown that such statistics are sensitive to changes in texture structure, but not sensitive to changes in mean and variance. Therefore, it is good to apply it to those images in which the average does not change, and the correlation properties change from one segment to another. At the same time, the advantage of statistics is that it does not require the evaluation of any parameters. Such nonparametric statistics often turn out to be the most suitable for real situations. To segment real images using likelihood ratio statistics, it is necessary to be able to calculate sample correlation characteristics obtained from homogeneous sections of this image. A white rectangle describing it is built around this contour, defining the minimum and maximum coordinates. An example of such a contour is shown in Fig. 2.1. According to the data inside this contour, the program automatically calculates the sample mean, variance and correlation characteristics according to a given algorithm. For clarity, the program plots correlation functions, allowing the operator to assess by eye the degree of difference in characteristics in different areas of the image. Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) PROCESSING OF AEROSPACE INFORMATION IN IMAGE CONTOURING PROBLEM 23 Fig. 2.1. Real image. Correlation characteristics using the Yule-Walker equations are recalculated into the coefficients of equation (2.3), and the latter, in turn, are substituted into the likelihood ratio statistics. The result of image segmentation is shown in Fig. 2.2. Of particular interest is the Fig. 2.2. Segmentation result. possibility of applying the developed algorithms and programs to radar images of the sea surface. The sensitivity of the algorithms to small changes in the surface structure allows detecting and detecting the form of anomalies on the windy surface caused by various natural and man-made sources [9, 17, 24]. 2.2. Statistical and Information Approaches There is an alternative approach for image texture analysis which is based on statistical characteristics of the image. This approach uses different so called textural features based on gray-level co-occurrence matrix. Gray-level co-occurrence matrix (GLCM) method have been first described by Julesz [13], where gray level co-occurrence matrix as the frequency of the appearance of individual pairs of values in a specific image fragment was introduced. The calculations of the element of normalized symmetrical GLCM p(i, j) are based on the analysis of quantized levels of brightness I p(i, j) = |{(n,m), (k, l) : I(n,m) = i, I(k, l) = j}|, (2.6) Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) 24 A.A. GALYAEV, A.V. DOBROVIDOV, P.V. LYSENKO, L.M. BERLIN, M.A. KOLYGO where I(n,m) are the values of the brightness matrix elements with coordinates (n,m), i, j are the quantized levels of brightness (i, j = 1, . . . , N ), p(i, j) is the frequency value of the pair having index (i, j). According this matrix statistical features concerning certain texture might be calculated. Most of these functions were suggested by Haralick et al. [10]. Table 2.1 contains known popular textural features [4,15] as well as two new ones, which are introduced in this article: so called statistical complexities CSQ and CTV [2, 3, 18]. no. Textural feature Formula 1 Energy ∑ i,j p(i, j)2 2 Contrast ∑ i,j p(i, j)(i− j)2 3 Correlation ∑ i,j p(i, j) (i− µ)(j − µ) σ2 4 Inertia ∑ i,j p(i, j)(i, j)2 5 Entropy − ∑ i,j p(i, j) log2 p(i, j) (0 if p(i, j) = 0) 6 Complexity CSQ ( − ∑ i,j p(i, j) log2 p(i, j) )(∑ i,j (p(i, j)− pw(i, j)) 2 ) 7 Complexity CTV ( − ∑ i,j p(i, j) log2 p(i, j) )(∑ i,j |p(i, j)− pw(i, j)| )2 Table 2.1. Most common features used in textural image analysis. Explanations for Table 2.1: pw(i, j) – the normalized GLCM for the brightness levels matrix I distributed according to the normal distribution for white noise simulation; µ is the mean GLCM (which is an estimate of the intensity of all pixels in the relationship that contributed to the GLCM), calculated as: µ = ∑ i,j ip(i, j); σ2 is the variance of the intensities of all reference pixels in the relationship contributing to the GLCM is calculated as: σ2 = ∑ i,j p(i, j)(i− µ)2. 3. MODELLING AND COMPARISON OF CONSTRUCTED METHODS Two methods described in article are compared for different satellite image examples in this Section. Fast Python implementation of GLCM has been used from [11] and modified for complexities functions’ computation (items 6 and 7 in Table 2.1). Markov field algorithm has been implemented by authors. 3.1. Satellite Images Contouring The first example is devoted solely to image contouring problem itself. Two satellite images are used. Both are taken from ESA ”Earth from Space image collection” gallery: the first one Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) PROCESSING OF AEROSPACE INFORMATION IN IMAGE CONTOURING PROBLEM 25 is ESA Sentinel-2 picture of Faroe Islands [7]; the second one is ESA Sentinel-1 picture of Lofoten archipelago in Norway [8]. These pictures have been chosen due to their extensive map ruggedness and contour line complexity as a perfect material for contouring algorithms validation. Two methods of textural image analysis are applied to each picture, which is illustrated in Figures 3.3 and 3.4. It can be seen in Figure 3.3 that image contouring problem is solved successfully with both methods in the first example. 0 200 400 600 800 1000 0 200 400 600 800 1000 1200 (a) Original picture. 0 200 400 600 800 1000 0 200 400 600 800 1000 1200 (b) Processed with Markov field algorithm picture. 0 200 400 600 800 1000 0 200 400 600 800 1000 1200 (c) Processed with GLCM complexity algorithm picture. Fig. 3.3. The image of Faroe Islands by Copernicus Sentinel-2 mission. 0 500 1000 1500 2000 0 500 1000 1500 2000 2500 3000 (a) Original picture. 0 500 1000 1500 2000 0 500 1000 1500 2000 2500 3000 (b) Processed with Markov field algorithm picture. 0 500 1000 1500 2000 0 500 1000 1500 2000 2500 3000 (c) Processed with GLCM complexity algorithm picture. Fig. 3.4. The image of the archipelago of Lofoten by Copernicus Sentinel-1 mission. Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) 26 A.A. GALYAEV, A.V. DOBROVIDOV, P.V. LYSENKO, L.M. BERLIN, M.A. KOLYGO Figure 3.4 contains a much more difficult image due to windy sea surface. It can be seen that GLCM complexity approach performs better than Markov field method, denoising slightly more of clouds and sea surface. 3.2. Ship Detection Second example studies a more practical problem of object detection on satellite images. The first and second picture is taken from [17] for research purposes. Figure 3.5 demonstrates the image of many anchored static ships. As one can see, both algorithms perform satisfactorily in this example, although Markov field algorithm loses a little more pixels of ships. 0 200 400 600 800 1000 0 200 400 600 800 1000 (a) Original picture. 0 200 400 600 800 1000 0 200 400 600 800 1000 (b) Processed with Markov field algorithm picture. 0 200 400 600 800 1000 0 200 400 600 800 1000 (c) Processed with GLCM complexity algorithm picture. Fig. 3.5. Anchored cargo ships from [17]. Figure 3.6 explores a more challenging case of moving ship. In this situation there can be also a problem of ship wake detection as studied in [24]. As one can see from figures 3.6b and 3.6c, both algorithms successfully detect a moving object but second one allows also to detect a wake of the ship, which can be useful in different applications. 0 200 400 600 800 1000 0 200 400 600 800 1000 (a) Original picture. 0 200 400 600 800 1000 0 200 400 600 800 1000 (b) Processed with Markov field algorithm picture. 0 200 400 600 800 1000 0 200 400 600 800 1000 (c) Processed with GLCM complexity algorithm picture. Fig. 3.6. The internal waves generated by a cargo ship in the Pearl River estuary in China from [17]. Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) PROCESSING OF AEROSPACE INFORMATION IN IMAGE CONTOURING PROBLEM 27 4. CONCLUSION The article proposes a solution to the problem of segmentation of real satellite images based on random Markov vector fields and on the basis of new textural features. The authors propose to use two types of statistical complexity that distinguish deterministic structures well on naturally or artificially noised images. On the example of several satellite images the two presented methods are compared and conclusions are drawn about their applicability depending on the imaging conditions and complexity of the depicted scene as well as the prospects of their use in automatic image processing are discussed. Future work can be devoted to implementation of classification algorithms based on textural features introduced above. ACKNOWLEDGEMENTS The work was supported by the Russian Science Foundation under grant no 23-19-00134. REFERENCES 1. Benecki, P., Kawulok, M., Kostrzewa, D. & Skonieczny, L. (2018) Evaluating super- resolution reconstruction of satellite images, Acta Astronautica, 153, 15–25. 2. Berlin, L., Galyaev, A. & Lysenko, P. (2023) Comparison of Information Criteria for Detection of Useful Signals in Noisy Environments, Sensors, 23(4). 3. Berlin, L., Galyaev, A. & Lysenko, P. (2023) Statistical complexity as a criterion for the useful signal detection problem, Automation and Remote Control, 84(7). 4. Bharati, M. H., Liu, J. J. & MacGregor, J. F. (2004) Image texture analysis: methods and comparisons, Chemometrics and Intelligent Laboratory Systems, 72(1), 57–71. 5. Christodoulou, C. I., Michaelides, S. C. & Pattichis, C. S. (2003) Multifeature texture analysis for the classification of clouds in satellite imagery, IEEE Transactions on Geoscience and Remote Sensing, 41(11), 2662–2668. 6. Ermakov, S. A., Lavrova, O. Yu., Kapustin, I. A., Ermoshkin, A. V., Molkov, A. A. & Danilicheva, O. A. (2018) O grebenchatoi strukture granits slikov na morskoi poverhnosti [On the “comb” structure of the edges of slicks on the sea surface], Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 15(7), 208– 217, [in Russian]. 7. ESA (2018). Faroe Islands, Modified Copernicus Sentinel data, [Online]. Available: https://www.esa.int/ESA Multimedia/Images/2020/01/Faroe Islands. 8. ESA (2020). Lofoten, Norway, Modified Copernicus Sentinel data, [Online]. Available: https://www.esa.int/ESA Multimedia/Images/2022/03/Lofoten Norway 9. Graziano, M. D. (2020) Preliminary Results of Ship Detection Technique by Wake Pattern Recognition in SAR Images, Remote Sensing, 12(18), 2869. 10. Haralick, R. M. (1979) Statistical and structural approaches to texture, Proceedings of the IEEE, 67(5), 786–804. 11. Izumi, T. (2018). Fast Gray-Level Co-Occurrence Matrix by numpy, [Online]. Available: https://github.com/tzm030329/GLCM. 12. Jahjah, M. & Ulivieri, C. (2010) Automatic archaeological feature extraction from satellite VHR images, Acta Astronautica, 66(9), 1302–1310. 13. Julesz, B. (1962) Visual Pattern Discrimination, IRE Transactions on Information Theory, 8(2), 84–92. 14. Keeler, R. N., Bondur, V. G. & Gibson C. H. (2005) Optical satellite imagery detection of internal wave effects from a submerged turbulent outfall in the stratified ocean: optical satellite imagery detection, Geophysical Research Letters, 32(12). Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) https://www.esa.int/ESA_Multimedia/Images/2020/01/Faroe_Islands https://www.esa.int/ESA_Multimedia/Images/2022/03/Lofoten_Norway https://github.com/tzm030329/GLCM 28 A.A. GALYAEV, A.V. DOBROVIDOV, P.V. LYSENKO, L.M. BERLIN, M.A. KOLYGO 15. Kiema, J. B. K. (2002) Texture analysis and data fusion in the extraction of topographic objects from satellite imagery, International Journal of Remote Sensing, 23(4), 767– 776. 16. Kupidura, P. (2019) The Comparison of Different Methods of Texture Analysis for Their Efficacy for Land Use Classification in Satellite Imagery. Remote Sensing, 11(10), 1233. 17. Liu, Y. & Deng, R. (2018) Ship Wakes in Optical Images, Journal of Atmospheric and Oceanic Technology, 35(8), 1633–1648. 18. López-Ruiz, R., Mancini, H. L. & Calbet, X. (1995) A statistical measure of complexity, Physics Letters A, 209(5), 321–326. 19. Napolano, G., Vela, C., Nocerino, A., Opromolla, R. & Grassi, M. (2023) A multi-sensor optical relative navigation system for small satellite servicing, Acta Astronautica, 207, 167–192. 20. Pan, X., Li, N., Yang, L., Huang, Z., Chen, J., Wu, Z. & Zheng, G. (2023) Anomaly- Based Ship Detection Using SP Feature-Space Learning with False-Alarm Control in Sea-Surface SAR Images, Remote Sensing, 15(13), 3258. 21. Pesaresi, M. (2000) Texture Analysis for Urban Pattern Recognition Using Fine-resolution Panchromatic Satellite Imagery, Geographical and Environmental Modelling, 4(1), 43–63. 22. Sun, C. & Wen, Z. (2021) Research Progress in Remote Sensing Image Texture Analysis, Journal of Physics: Conference Series, 1961(1), 012028. 23. Sun, Z., Sun, S., Zhao, J., Ai, B. & Yang, Q. (2022) Detection of Massive Oil Spills in Sun Glint Optical Imagery through Super-Pixel Segmentation, Journal of Marine Science and Engineering, 10(11), 1630. 24. Wang, J., Guo, L., Wei, Y. & Chai, S. (2023) Study on Ship Kelvin Wake Detection in Numerically Simulated SAR Images, Remote Sensing, 15(4), 1089. Copyright © 2024 ASSA. Adv Syst Sci Appl (2024) Introduction Information model of spatial interaction Markov Model Approach Statistical and Information Approaches Modelling and comparison of constructed methods Satellite Images Contouring Ship Detection Conclusion