Начиная с начала 2000 года осуществляется внедрение GHIS в здравоохранении, в рамках принятого проекта о реформирование информ Mathematical Problems of Computer Science 44, 59--66, 2015. Video-Based Automated System for Pavement Surface Quality Monitoring Gurgen H. Hakobyan National Polytechnic University of Armenia e-mail: hakobyan.gurgen@yahoo.com Abstract Nowadays, an important task is to evaluate the pavement condition, in order to perform necessary works for keeping the road safe. Performing such kind of surveys and calculations on the roads only by humans is ineffective not only concerning time but it also requires more costs and may be unsafe for pavement condition inspectors. In this paper we introduce an automated monitoring system, which can classify cracks and artifacts from pavement video images and illustrate the processed images results during image acquisition. Firstly, a threshold value is counted using the image histogram. Then, image processing operations are applied, namely binarization and segmentation on each frame. Afterwards, the image is filtered from small segments which don't contain crack information. Finally, the defect percent of each frame is counted and displayed on the graph real-time, where areas with cracks are distinguishable for pavement management operators. This information is helpful for pavement condition rating. Experimental data were from real road images in Armenia. Keywords: Pavement, Crack, Automated quality monitoring, Binarization, Segmentation. 1. Introduction In pavement management there is always a need of surface quality assessment. It requires image acquisition, image processing, distress evaluation. Before 1980, all these inspections were accomplished manually [1]. Doing these kinds of surveys and calculations on the roads only by humans is ineffective not only concerning time but it also requires more costs and may be unsafe for pavement condition inspectors. Also this method of field inspection poses several drawbacks, such as: slow, labour intensive and expensive, subjective approach generating inconsistencies and inaccuracies in the determination of pavement condition; inflexible and not providing an absolute measure of the surface, it has poor repeatability since the assessment of the given pavement section may differ from one survey to another, could expose a serious safety hazard to 59 Video Based Automated System for Pavement Surface Quality Monitoring 60 the surveyors due to high speed and high volume traffic [2]. Thus, over the last decades, a demand of automated digital imaging systems came out for pavement quality monitoring. Nowadays, this task can be automated with noninvasive techniques to be more comfortable, less dangerous for employees and users of the road but also more efficient and less expensive than manual methods [1]. Also with an automated image processing system, processing of survey images may be done more accurately and objectively. Pavement quality is characterized by many factors, including the surface deflection from regulated standards as well as different types of defects, caused by regular pavement exploitation. Crack is the most common distress type of asphalt pavement, which is classified in 4 main types: longitudinal (linear), block, transverse, alligator. Types of cracks are shown in Figure 1. Crack has three characteristics: darker than its surrounding (because of crack form, many light rays cannot be reflected from crack to camera); continuous (crack can be thinner than aggregates but it is always longer than them); dominant orientation (on all length of crack or on each segment) [3]. Longitudinal Transverse Block Alligator Fig. 1. Main types of pavement cracks. Crack detection is a difficult task, because most of the time it is represented weakly on the image and depending on pavement texture, may be low contrasted. In case of low contrast, pre- processing operations are needed, such as image normalization and equalization. There are various automated systems for crack detection, assessment, classification [1-5]. A typical pavement management system consists of image data collection, verification, processing and analysis. In [3] an automatic crack detection and classification are introduced, which can detect small cracks as 1mm in width, joint and bridged types of cracks. First, pre-processing is applied to images to remove lane-marking. Then an anisotropy measure is calculated to detect road defects. Finally, a back-propagation neural network is used to classify the images into four classes: defect-free, crack, joint and bridged. Although, in that work, the road images are labelled by human operators. In [4] a method for automated detection and assessment of potholes, cracks and patches from video clips of highways is proposed. In that method, potholes, cracks and patches are detected and quantified automatically using CDDMC algorithm. Several devices are also developed, which monitor condition of road surface speedily and with better quality of data. Known examples are CSIRO’s road crack detection vehicle, and Roadware’s WiseCrax, crack detection system. Unfortunately, by the commercial nature of these systems, information on their algorithms of image processing for the crack detection is limited [6]. As described above, the main difficulty of effective detection of abnormal pavement areas (in particular - cracks) is the variability of image characteristics (such as brightness, contrast, etc.). So, any method of automated pavement abnormal area detection should have enough flexible algorithms, which will be adaptable for concrete conditions under which the monitoring is executed. Previously, we developed an image morphological analysis technique [7], which allows to classify cracks and artifacts on each frame by using special procedures. G. Hakobyan 61 In this paper we propose an automated pavement quality monitoring system, by using statistical analysis methods on pavement image segments. With this system pavement manage- ment operators will be able to easily detect distressed areas real-time and rate the pavement quality. 2. Image Processing Procedure for Pavement Quality Assessment The main ideology for pavement quality assessment, by means of photo or video processing, relies on researches of morphological properties of processed image. This kind of processing includes known algorithms of binarization, segmentation, edge detection, etc. Fig. 2. Work concept of automated system and image acquisition vehicle. Figure 2 illustrates a short scheme which includes the main steps of proposed procedure for pavement quality assessment. The proposed monitoring method is based on the following operations: image acquisition, threshold selection, binarization, segmentation, filtration, crack detection, pavement quality assessment. Short descriptions of the mentioned procedures are given below. Image acquisition. Images are collected from the roads in Armenia with size 320x240. The camera is mounted on the special car (as it is shown in Fig. 2) which contains a mounted camera on it and a powerful light illumination which will make image pre-processing operations, such as normalization, equalization, etc., easier. Depending on driving speed the frequency of shots is counted the way that all images together cover the whole canvas of pavement. In order to apply our image processing methods each frame is converted into 8 bit grayscale image, processed live and also stored in the database. The captured data is sent to image processing software, which is described in the next sections. Threshold selection. Usually, cracks and potholes in pavement images are characterized by dark intensity values, which are also noticeable from histograms. In order to detect them we use image processing methods such as binarization and segmentation. For image binarization we Image acquisition Threshold selection Binarization and segmentation Crack detection Pavement quality assessment Segments filtration Video Based Automated System for Pavement Surface Quality Monitoring 62 need to find an optimal threshold value, for extraction of foreground in the image. An example of pavement video images with and without a crack and their histograms are shown in Figure 3. As it is visible on histograms (c, d), pavement images contain only one mode, which makes it almost impossible to separate cracks from background using the known threshold selection methods (such as Otsu method [8]). a b c d Fig. 3. Images with and without crack and their histogram. There is a small difference between histograms with and without cracks, but it's not possible to definitely detect cracks just by histogram difference, especially during automated image processing. In this paper, we count threshold using bα generalized parameter, which defines the portion of "black" pixels inside the image pixels array after binarization. By using the bα parameter we preliminarily fixate the array of pixels, which are connected with each other and form all the "black" segments of the image. Depending on pavement structure those segments will be classified as cracks or artifacts. Binarization, segmentation and filtration. Following [7], on each frame using its bt threshold, we apply binarization. The bt threshold is counted according to bα value from pixels intensity histogram of the relevant image. Then full segmentation is applied to binary image. A segment is considered an array of "black" pixels each of which has at least one neighbor pixel from the same array. Notice that in the end the problem of segments classification boils down into search of threshold value of segments sizes - sT , which will split the segments into cracks and artifacts. Segments which have less pixels than sT in total characterize the pavement surface which doesn't contain cracks, the rest characterize cracks on the pavement. Threshold value sT may be counted in a couple of ways depending on various conditions. In the paper the simple method of "three-sigma limits" is used, i.e., 𝑇𝑇𝑠𝑠 = �̂�𝜇𝑏𝑏 + 3𝜎𝜎�𝑏𝑏 formula is applied, where �̂�𝜇𝑏𝑏 and 𝜎𝜎�𝑏𝑏 are values of mean and standard deviation of segment sizes. After counting sT , the segments smaller than that value are removed from the image. Crack detection and pavement quality assessment. After these operations we have all the necessary information for counting crack area inside each frame (D%) ,100*% N segKD = G. Hakobyan 63 where segK is the total count of "black" pixels of all segments after filtration, N - count of all pixels of the image. After applying this processing on all images of video sequence and adding D% to the chart, we can monitor and distinguish areas which contain cracks. A. Developed software An image processing software is developed based on binarization and segmentation algorithms [9]. It can be used for crack detection and assessment from static images [7] as well as for processing video stream images of pavement and its quality monitoring. Currently, the following operations are available: extract pixels matrix from images, build image from pixels matrix, automated threshold selection based on selected α value, binarization, segmentation, segments filtration based on minimum and maximum input values of segments, results exportation(.xls(x), .csv) as well as video stream images processing and building live chart which shows defect percent of each frame. All processing operations require 8 bit grayscale images. Interface of the software is shown on Figure 4. Video stream processing example results are shown on the next section. During video images processing, the software is able to send each result data chunk to remote server. It can be useful if there is a need for a remote data examination or it is required to see the results from more than one location. The image processing may be done in the following ways:  Static image processing. If bα is defined, binarization threshold and segments min-length for filtration (described above) are counted automatically. It's also possible to input all those values manually. Software will output detected crack, total count of segments and defect percent.  Video stream processing. Takes input parameter bα , outputs each frame detected crack image, defect percent and builds live chart (Figure 5) and stores results in the database. For more accuracy it's also possible to change the factor value 𝜎𝜎�𝑏𝑏. The software was developed using Java programming language. Fig. 4. Software state during processing. Video Based Automated System for Pavement Surface Quality Monitoring 64 3. Experimental Results For illustration of achieved results a chart is shown in Figure 5, which represents D% values of 128 images from the same video sequence. The chart is being built on-time automatically during image acquisition and processing. As you can see from the chart, D% takes smaller values on the first images and high values on the last images. Those values represent small and big cracks, respectively. Depending on intensity variations on some of the shots there can be several jumps on the charts, which can be ignored. Fig. 5. Pavement monitoring chart. The corresponding image examples are shown in Table 1 (row 3). On the first row of the table frame numbers are shown. On the second row the distances from the beginning of image acquisition are shown. Images after all steps of processing described above (binarization, segmentation, small segments filtration) and defect percent values of each frame are shown on the fourth and fifth rows, respectively. Table 1. Processed data during monitoring. Frame N 1 20 40 60 82 90 100 110 120 Distance (m) 0 2.5 5 7.5 10.25 11.25 12.5 13.5 15 Image Binarization, Segmentation , Filtration D(%) 0.03 0.2 0.02 0.04 0.2 0.9 1 0.7 0.3 G. Hakobyan 65 4. Conclusion Proposed automated system is based on image processing algorithms such as binarization and segmentation. It can be used in pavement management for surface condition monitoring. On real experimental data it is shown that by using this system it will be easy for pavement management operators to distinguish areas which contain cracks or other type of pavement distresses. References [1] S. Chambon and J. M. Molinard, “Automatic road pavement assessment with image processing: review and comparison", International Journal of Geophysics, vol. 2011, Article ID 989354 [2] M. Mustaffar, T. C. Ling and O. C. Puan, “Automated pavement imaging program (APIP) for pavement cracks classification and quantification - a photogrammetric approach”, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 37, Part B4, pp. 367-372, 2008 [3] T. S. Nguyen, M. Avila and S. Begot, “Automatic detection and classification of defect on road pavement using anisotropy measure”, 17th European Signal Processing Conference, pp. 617-624, 2009. [4] L. Huidrom, L. K. Das and S. K. Sud, “Method for automated assessment of potholes, cracks and patches from road surface video clips”, 2nd Conference of Transportation Research Group of India, pp. 312-321, 2013. [5] L. Li, L. Sun, G. Ning and S. Tan, “Automatic pavement crack recognition based on bp neural network, swarm intelligence”, Promet – Traffic&Transportation, Vol. 26, pp. 11- 22, 2014 [6] T. Sy Nguyen, M.Avila, S. Begot and J.-C. Bardet, “Detection of defects in road surface by a vision system”, Electrotechnical Conference, The 14th IEEE Mediterranean, AJACCIO, France, pp.847 – 851, 2008. [7] Д. Г. Асатрян и Г. O. Акопян, “Методика морфологического анализа изображения и оценивания качества дорожного покрытия”, В е с т н и к Российско-Армянского (Славянского) университета, №1, pp. 36-44, 2015. [8] N. Otsu, “A threshold selection method from gray-level histograms”, IEEE Trans. Syst. ManCybern, vol.9, pp. 62–66, 1979. [9] D. G Asatryan, G. S. Sazhumyan and H. S. Shahverdyan, “Technique for coherent segmentation of image and applications”, Transactions of IIAP of NAS RA, Mathematical Problems of Computer Science, vol. 28, pp. 88–93, 2007. Submitted 04.08.2015, accepted 24.11.2015 Video Based Automated System for Pavement Surface Quality Monitoring 66 Տեսանկարահանման վրա հիմնված ավտոմատացված համակարգ՝ ճանապարհային ծածկույթի որակի հսկում իրականացնելու համար Գ. Հակոբյան Ամփոփում Մեր օրերում կարևոր խնդիր է ճանապարհային ծածկույթի որակի գնահատումը՝ ճանապարհների անվտանգությունը և արդյունավետությունը ապահովելու նպատակով: Նման աշխատանքների իրականացումը առանց համապատասխան տեխնիկական միջոցների ժամանակատար է, պահանջում է մեծ ծախսեր և վտանգավոր է չափումներ իրականացնող անձնակազմի համար: Հոդվածում առաջարկվում է ավտոմատացված համակարգ, որը հնարավորություն է տալիս տեսանկարահանման օգնությամբ ճանապարհային ծածկույթի պատկերներից հայտնաբերել ծածկույթի արատները և համապատասխան մշակման միջոցով գնահատել ծածկույթի որակը: Արդյունքները արտապատկերվում են համակարգում տեղակայված համակարգչի էկրանի վրա: Автоматизированная система контроля качества дорожного покрытия на основе видеообработки Г. Акопян Аннотация В наши дни, для обеспечения качества и эффективности дорожного покрытия, важным вопросом является оценка состояния поверхности покрытия. Такого рода исследования без соответствующих технических средств требуют не только больше времени, но и больших затрат и могут быть опасными для специалистов, которые проводят исследования. В статье предлагается автоматизированная система контроля качества, которая дает возможность с помощью соответствующей обработки выявлять дефекты дорожного покрытия и оценить качество покрытия. Результаты отображаются на экране компьютера, который вмонтирован в предлагаемую автоматизированную систему.