Microsoft Word - ETASR_V11_N4_pp7291-7295 Engineering, Technology & Applied Science Research Vol. 11, No. 4, 2021, 7291-7295 7291 www.etasr.com Farooq et al.: Estimation of Traffic Occupancy using Image Segmentation Estimation of Traffic Occupancy using Image Segmentation Muhammad Umer Farooq Department of Computer Science and IT NED University of Engineering and Technology Karachi, Pakistan umerora1@gmail.com Shariq Mahmood Khan Department of Computer Science and IT NED University of Engineering and Technology Karachi, Pakistan shariq@neduet.edu.pk Afzal Ahmed Department of Urban and Infrastructure Engineering NED University of Engineering and Technology Karachi, Pakistan afzalahmed@neduet.edu.pk Muhammad Bilal Nawaz Department of Urban and Infrastructure Engineering NED University of Engineering and Technology Karachi, Pakistan mbilalnawaz91@gmail.com Abstract-Increased traffic flow results in high road occupancy. Traffic road occupancy is often used as a parameter for the prediction of traffic conditions by traffic engineers. Although traffic monitoring systems are based on a large number of technologies, challenges are still present. Most of the methods work efficiently for free-flow traffic but not in heavy congestion. Image processing techniques are more effective than other methods, as they are based on loop sensors and detectors to monitor road traffic. A huge number of image frames are processed in image processing hence there is a need for a more efficient and low-cost image processing technique for accurate vehicle detection. In this paper, a novel approach is adopted to calculate road occupancy. The proposed framework has robust performance under road conjunction and diverse environmental conditions. A combination of image segmentation threshold technique and shadow removal technique is used. The study comprised of segmenting 1056 images extracted from recorded videos. The obtained results by image segmentation were compared with traffic road occupancy calculated manually using Autocad. A final percentage difference of 8.17 was observed. Keywords-image segmentation; road occupancy; shadow removal I. INTRODUCTION Precise and accurate calculation of basic traffic flow parameters is a major step towards the successful planning and management of a traffic system. Real time traffic flow can be used to enhance the capacity of a traffic system. Traffic road occupancy is often used to determine traffic congestion and related characteristics. It is the ratio of the area occupied by vehicles to the total area of the road in a traffic scenario. Road occupancy has been a subject of interest for traffic management and planners over time [1, 2]. A consequence of traffic density is road congestion and the enhanced probability of accidents. There is an exponential increase in the number of total vehicles globally during the last decade. Traffic monitoring is a difficult task due to the high traffic occupancy and issues such as shadows, occlusion etc. For the advancement of traffic monitoring systems, different methods and techniques have been applied. However, obtaining traffic road occupancy for heterogeneous traffic (e.g. the traffic of the city of Karachi) is a difficult task due to the variety of traffic modes in the traffic composition, the ranging in area and size, and little to no lane discipline. So, a method needs to be developed for obtaining road occupancy [3]. Image processing has been used in a number of ways to obtain real time traffic in terms of both model flow and density. However, the accuracy of the models generated is highly traffic dependent and occlusion reduces greatly the accuracy of the models [4]. For heterogeneous traffic, occlusion affects accuracy and real time traffic management systems cannot be adopted [3-5]. A spatio-temporal Markov random field algorithm for traffic images at intersections was developed in [6]. This algorithm traces each pixel of the image on the x-y plane versus the time axis. The authors also developed an extended recognition system based on hidden Markov models. This system has the ability to learn and keep track of events, thus it can be used to identify events like illegal u-turns or reckless driving. A novel framework was developed to explore multidimensional data of road traffic to examine different patterns of traffic and anomaly detection in [3]. The framework was implemented on a collection of road traffic datasets gathered from different areas of the city. A three step method was ormulated for vehicle counting in images with multiple vehicle occlusions in [7]. The first step involves the deduction of vertices for each vehicle from the camera configuration. In the second step, a contour description model is used to determine the vanishing contours with their directions to determine vehicle counts, and finally, in the last step, a resolvability index is assigned to each occluded vehicle to find out the dimensions of the occluded vehicles. Another methodology was purposed [8] to perform video analysis for Corresponding author: Muhammad Umer Farooq Engineering, Technology & Applied Science Research Vol. 11, No. 4, 2021, 7291-7295 7292 www.etasr.com Farooq et al.: Estimation of Traffic Occupancy using Image Segmentation counting vehicles. The methodology involved the formulation of a bounding box to detect the vehicles and then the use of a Kalman filter and adaptive background subtraction to count vehicles. A study for the number plate recognition of cars moving on the road in Saudi Arabia was conducted in [9]. In the proposed mechanism, a camera captures the image of the car and uses image segmentation algorithms to obtain the special letters. This mechanism was tested on 610 images taken under different illumination conditions. Image segmentation plays an essential role in understanding of image, investigation of medical images, video surveillance, image compression, etc. [10-12]. The objective of image segmentation is the clustering of pixels into image regions. These regions refer to individual objects, surfaces or parts of objects. Most research has been done in the field of vehicle detection, vehicle tracking, vehicle recognition, and vehicle behavior understanding [4, 13]. Beside these, the estimation of traffic occupancy is also essential in improving traffic management systems. In this research, image segmentation is used for the estimation of traffic occupancy. For complex traffic scenarios, image processing techniques are playing an important role in a wide range of applications. Image segmentation deals with the image processing and refers to the segregation of an image into various regions based on texture, color, or intensity. Image segmentation can be performed by different techniques and algorithms. Among these techniques we have chosen the color threshold technique to obtain road occupancy. This technique does not require prior knowledge of the image, has simple calculations, and fast operating speed. The videos were recorded during daytime, therefore the shadows of vehicles cause inaccuracy in the results. This issue is tackled by adopting a novel combination approach of color threshold with shadow removal. Threshold segmentation algorithm Otsu, which is an interclass variance method is used [14]. The results showed that this approach minimizes the error due to the shadows of vehicles and leads to a better estimation of road occupancy. II. RESEARCH METHODOLOGY A. Otsu’s Thresholding Thresholding plays an important role in the segmentation of an image into various regions and in pattern recognition. Some grayscale values are selected automatically as threshold and the image is classified into different regions. Among various thresholding techniques, we have chosen Otsu’s thresholding method [14] for this research, which is a globally accepted threshold method. Through this method, maximum separation of classes can be obtained and histogram image thresholding can be performed automatically. It is assumed that in Otsu’s algorithm, an image consists of two classes, foreground (��) and background (��), and based on probability density function a normalized histogram is constructed. The probability density function is: �� �� � � � , � � 0,1,2,…. . ,� � 1 (1) where n is the total pixels of the image, � is the pixel having intensity level � , and � is the maximum intensity level of the image. Center point between the minimum and maximum intensity level in the image is known as the initial threshold. If � is the initial threshold, then the set of pixels ��� has levels �0,1,……. . ,m � 1� and the set of pixels ��� has levels �m,m � 1,……. . ,N� 1�. The extension of Otsu’s method for multilevel thresholding is given as: �� � !��"� �"# � � !��"� � "# � (2) where: !� � ∑ � �� %&� '� (3) !� � ∑ � �� (&� '% (4) "� � ∑ �� �� /!� %&� '� (5) "� � ∑ �� �� /!� *&� '� (6) "# � ∑ �� �� *&� '� (7) The optimal threshold value can be achieved by +�,�+-�� . B. Traffic Occupancy Estimation by Otsu’s Thresholding In order to perform image segmentation, videos were recorded on the project site, which is the University road of Karachi near the Federal Urdu University. For an accurate occupancy calculation the camera needs to be parallel with the road. The more the camera is titled beyond 180 0 angle, more error will be observed. Due to site limitation and in order to attain maximum height, the camera was installed on the 5 th floor of a building. The research methodology is shown in Figure 1. Videos were recorded for different times of the day in order to have samples from all traffic conditions for analysis. The videos were cropped to obtain frames having road surface and vehicles only. After the extraction of image frames, image segmentation was performed in Matlab. The obtained results were compared with manual road occupancy results obtained in Autocad. Finally, the results were displayed by plotting graphs of image segmented occupancy and original occupancy obtained from Autocad. Fig. 1. Proposed framework. C. Data Collection and Analysis Using Matlab, frames were extracted from videos at an interval of 3 frames per minute. The camera recorded the videos at 30fps (frames per second). Every minute had 1800 Engineering, Technology & Applied Science Research Vol. 11, No. 4, 2021, 7291-7295 7293 www.etasr.com Farooq et al.: Estimation of Traffic Occupancy using Image Segmentation frames, so the 600 th , 1200 th , and 1800 th frames of every minute were analyzed. Matlab offers multiple functions and tools to perform image segmentation. Original image and threshold image are shown in Figure 2. A binary filter was applied after thresholding to convert the image in two parts, background (black) and foreground (white). The mask created was exported to Matlab script where operations were defined including image import and applying the created mask. In order to render the small pixel areas left within the image, such as the windscreens of the cars, they were filled using the hole filling option. Finally, the total number of white and black pixels was obtained. A ratio of sum of all white pixels and a sum of all pixels gave the occupancy. (a) (b) (c) (d) Fig. 2. (a), (b) Original and (c), (d) segmented images. Fig. 3. Autocad calculating original occupancy values. In order to calculate the accuracy of the procedure, these images were imported to Autocad and the object areas were calculated by drawing a boundary around all vehicles in the image. Figure 3 is a pictorial presentation of the calculation of the original occupancy values. A ratio of the sum of the area of all the objects to the total area of the image was computed to calculate the total occupancy. D. Shadow Removal In order to minimize the errors due to the formation of shadows on the road by the passing vehicles, it is necessary to devise a technique for the shadow removal. Due to the resemblance of the shadows with the dark color objects in an image often the detection and removal of shadows turns out to be a difficult task. In order to remove shadows another mask was defined in Matlab. L*a*b color space is the tool used to define the shadow removal mask. Here, L* is for the lightness and a* and b* for the green–red and blue–yellow color components. Original and shadow detected images are shown in Figure 4. (a) (b) (c) (d) Fig. 4. (a), (b) Original and (c), (d) shadow detected segmented images. III. RESULTS A total of 1056 image frames were analyzed. Based on the comparison of the average of the real occupancy calculated with Autocad vs the occupancy calculated by Matlab, a percentage difference of 33.18 was observed before the application of shadow removal mask between the actual and Matlab occupancy. After the shadow removal mask the observed difference percentage was found to be 8.17. The results are given in Table I whereas Figure 5 exhibits the difference percentage between the calculated and the observed occupancy. During the research, a large difference percentage was observed for a few frames. This can be explained by the small value of occupancy in these frames. Even a small difference between the two occupancy values can result in a considerable percentage difference. IV. VALIDATION In order to validate the masking techniques proposed in this research, multiple frames were extracted. The results for the difference percentage in occupancy obtained by the masking techniques showed an average difference of 8.17 with 19.156 maximum and 0.906 minimum difference obtained. Engineering, Technology & Applied Science Research Vol. 11, No. 4, 2021, 7291-7295 7294 www.etasr.com Farooq et al.: Estimation of Traffic Occupancy using Image Segmentation (a) (b) (c) (d) (e) (f) (g) (h) Fig. 5. Difference between actual and image segmented road occupancy. TABLE I. RESULT COMPARISON OF TRAFFIC OCCUPANCY OBTAINED FROM IMAGE SEGMENTATION WITH AND WITHOUT SHADOW REMOVAL Image name Area of objects by autocad Area of individual image Actual road occupancy Occupancy by image segmentation without shadow removal Difference Occupancy by image segmentation after shadow removal Difference 600 0.060 0.343 17.539 21.706 23.761 18.689 6.55 1200 0.057 0.343 16.608 21.366 28.647 16.810 1.21 1800 0.074 0.343 21.785 27.317 25.390 19.670 -9.70 2400 0.056 0.343 16.433 13.913 -15.338 15.089 -8.18 3000 0.036 0.343 10.529 13.781 30.883 12.082 14.746 3600 0.077 0.343 22.484 17.289 -23.102 20.635 -8.223 4200 0.070 0.343 20.418 23.861 16.859 17.634 -13.638 4800 0.070 0.343 20.418 25.128 23.062 18.785 -7.997 5400 0.079 0.343 23.240 21.249 -8.567 19.861 -14.540 6000 0.062 0.343 18.091 23.017 27.227 19.846 9.698 6600 0.045 0.343 13.118 16.633 26.795 15.147 15.473 7200 0.058 0.343 16.986 17.337 2.063 15.938 -6.170 7800 0.040 0.343 11.721 14.305 22.037 13.238 12.934 8400 0.071 0.343 20.564 24.648 19.861 18.466 -10.202 9000 0.049 0.343 14.252 16.643 16.778 14.641 2.7286 9600 0.051 0.343 14.950 21.641 44.752 17.624 17.887 10200 0.068 0.343 19.982 17.990 -9.970 20.183 1.006 10800 0.062 0.343 18.179 21.314 17.247 18.215 0.198 11400 0.064 0.343 18.848 20.038 6.313 17.825 -5.426 12000 0.057 0.343 16.783 23.327 38.992 20.406 21.591 12600 0.080 0.343 23.356 21.914 -6.176 20.654 -11.570 13200 0.081 0.343 23.676 27.571 16.449 24.417 3.130 13800 0.075 0.343 22.047 23.497 6.573 19.817 -10.115 14400 0.103 0.343 30.046 32.189 7.133 27.933 -7.033 15000 0.068 0.343 19.837 22.631 14.087 20.795 4.831 15600 0.072 0.343 20.942 23.478 12.107 21.581 3.053 16200 0.078 0.343 22.833 25.101 9.935 21.300 -6.713 V. CONCLUSION Based on the observed results, it can be concluded that image segmentation can be effectively used to calculate the occupancy of the road segments, while being a cost effective solution. Results can be obtained at a very quick interval compared to the convenient methods of finding occupancy. Moreover, the occlusion will have minimal effect on the calculated occupancy. It is worth mentioning that the camera angle with the road surface plays an important role on the accuracy. The closer is the angle with the road to 180 0 and the placed camera in the middle lane, the better the result will be. REFERENCES [1] M. Won, "Intelligent Traffic Monitoring Systems for Vehicle Classification: A Survey," IEEE Access, vol. 8, pp. 73340–73358, 2020, https://doi.org/10.1109/ACCESS.2020.2987634. Engineering, Technology & Applied Science Research Vol. 11, No. 4, 2021, 7291-7295 7295 www.etasr.com Farooq et al.: Estimation of Traffic Occupancy using Image Segmentation [2] N. K. Jain, R. K. Saini, and P. Mittal, "A Review on Traffic Monitoring System Techniques," in Soft Computing: Theories and Applications, Singapore, 2019, pp. 569–577, https://doi.org/10.1007/978-981-13- 0589-4_53. [3] M. Umer, N. Ahmed, and M. Ali, "Unsupervised Video Surveillance for Anomaly Detection of Street Traffic," International Journal of Advanced Computer Science and Applications, vol. 8, no. 12, pp. 270– 275, Jan. 2017, https://doi.org/10.14569/IJACSA.2017.081234. [4] B. Tian et al., "Hierarchical and Networked Vehicle Surveillance in ITS: A Survey," IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 1, pp. 25–48, Jan. 2017, https://doi.org/10.1109/TITS. 2016.2552778. [5] A. Detho, S. R. Samo, K. C. Mukwana, K. A. Samo, and A. A. Siyal, "Evaluation of Road Traffic Accidents (RTAs) on Hyderabad Karachi M-9 Motorway Section," Engineering, Technology & Applied Science Research, vol. 8, no. 3, pp. 2875–2878, Jun. 2018, https://doi.org/ 10.48084/etasr.1920. [6] S. Kamijo, Y. Matsushita, K. Ikeuchi, and M. Sakauchi, "Traffic monitoring and accident detection at intersections," in Proceedings 199 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems (Cat. No.99TH8383), Tokyo, Japan, Oct. 1999, pp. 703–708, https://doi.org/10.1109/ITSC.1999.821147. [7] C. C. C. Pang, W. W. L. Lam, and N. H. C. Yung, "A Method for Vehicle Count in the Presence of Multiple-Vehicle Occlusions in Traffic Images," IEEE Transactions on Intelligent Transportation Systems, vol. 8, no. 3, pp. 441–459, Sep. 2007, https://doi.org/10.1109/ TITS.2007.902647. [8] E. Bas, A. M. Tekalp, and F. S. Salman, "Automatic Vehicle Counting from Video for Traffic Flow Analysis," in 2007 IEEE Intelligent Vehicles Symposium, Istanbul, Turkey, Jun. 2007, pp. 392–397, https://doi.org/10.1109/IVS.2007.4290146. [9] M. Sarfraz, M. J. Ahmed, and S. A. Ghazi, "Saudi Arabian license plate recognition system," in 2003 International Conference on Geometric Modeling and Graphics, 2003. Proceedings, London, UK, Jul. 2003, pp. 36–41, https://doi.org/10.1109/GMAG.2003.1219663. [10] J. N. Saeed, "A Survey of Ultrasonography Breast Cancer Image Segmentation Techniques," Academic Journal of Nawroz University, vol. 9, no. 1, pp. 1–14, Feb. 2020, https://doi.org/10.25007/ajnu. v9n1a523. [11] W. S. Chowdhury, A. R. Khan, and J. Uddin, "Vehicle License Plate Detection Using Image Segmentation and Morphological Image Processing," in Advances in Signal Processing and Intelligent Recognition Systems, Manipal, India, 2018, pp. 142–154, https://doi.org/10.1007/978-3-319-67934-1_13. [12] S. Bargoti and J. P. Underwood, "Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards," Journal of Field Robotics, vol. 34, no. 6, pp. 1039–1060, 2017, https://doi.org/ 10.1002/rob.21699. [13] K. K. Santhosh, D. P. Dogra, and P. P. Roy, "Anomaly Detection in Road Traffic Using Visual Surveillance: A Survey," ACM Computing Surveys, vol. 53, no. 6, pp. 119:1-119:26, Dec. 2020, https://doi.org/ 10.1145/3417989. [14] F. Garcia-Lamont, J. Cervantes, A. López, and L. Rodriguez, "Segmentation of images by color features: A survey," Neurocomputing, vol. 292, pp. 1–27, May 2018, https://doi.org/10.1016/j.neucom. 2018.01.091.