87 IMAGE ENHANCEMENT ON OBJECT DETECTION USING L0 GRADIENT PRIOR Sunario Megawan1*), Hernawati Gohzali2, Apriyanto Halim3 Program Studi Teknik Informatika Universitas Mikroskil https://www.mikroskil.ac.id/ 1*)sunario@mikroskil.ac.id, 2hernawati.gohzali@mikroskil.ac.id, 3apriyanto.halim@mikroskil.ac.id (*) Corresponding Author Abstrak Pendeteksian objek merupakan teknik yang digunakan untuk mengambil bagian-bagain tertentu pada citra. Bagian tersebut dapat berupa pemandangan, manusia atau benda-benda lainnya. Pada saat pendeteksian objek, citra yang didapatkan dapat mengalami penurunan kualitas citra yang dapat diakibatkan dari faktor cuaca, yaitu kabut, asap, debu, hujan dan lainnya. Penurunan kualitas pada citra, dapat mengakibatkan kesalahan pada klasifikasi dan tidak mampuan dalam mengenali objek pada citra. Oleh karena itu, proses perbaikan kualitas citra menjadi sangat penting untuk dilakukan pada saat tahap pre-processing dalam pendeteksian objek citra. Fokus masalah yang akan diselesaikan pada penelitian ini adalah pengembali an citra kabur dengan menggunakan L0 Gradient Prior. Hasil penelitian menunjukan penerapan L0 Gradient Prior dalam mengembalikan citra yang kabur dapat meningkat jumlah objek yang dapat diditeksi oleh sistem penditeksian objek. Kata kunci: Peningkatan kualitas citra, deteksi objek Abstract Object detection is a technique used to retrieve certain parts of the image. The part can be in the form of scenery, people, or other objects. At the time of object detection, the image obtained can experience a decrease in image quality which can be caused by weather factors, namely fog, smoke, dust, rain, and others. A decrease in the quality of the image can result in errors in classification and the inability to recognize objects in the image. Therefore, the process of improving image quality becomes very important to do at the pre-processing stage in detecting image objects. The focus of the problem to be solved in this study is the return of a blurred image using L0 Gradient Prior. The results showed that the application of L0 Gradient Prior in rest oring a blurred image can increase the number of objects that can be detected by the object detection system. Keywords: Image Enhancement, object detection INTRODUCTION Object detection is a technique used to retrieve certain parts of the image. The part can be in the form of scenery, humans or other objects (Vidal, Banerjee, Grm, Struc, & Scheirer, 2018). When performing object detection, the image obtained can experience a decrease in image quality which can be caused by weather factors, like fog, smoke, dust, rain, and others (Roy & Bhowmik, 2021). Decrease in image quality can also occur due to the process of increasing the size of the object in the image, thus making the image blurry or unclear. As for the consequences that can occur due to a decrease in image quality, namely errors in image classifiers or object suitability (Borel-Donohue & Young, 2019). Image quality degradation can also result in changes to information. Changes in information that are too large of course result in a lot of information being lost in the image. Image quality degradation may cause the system to not detect objects properly (Hasirlioglu, Reway, Klingenberg, Riener, & Huber, 2019). This is of course very necessary for further processes in the form of object detection, angle detection and others. Measurement of the level of change in the image before and after improving image quality can be seen from the success of object detection in the image. The method used in this research for object detection in the image is the Faster R-CNN algorithm (Ren, He, Girshick, & Sun, 2017). Faster R- CNN is the latest region-based generic object detection method that shows excellent results in various object detections (Wu, Yin, Wang, & Xu, 88 2019). Several studies have shown that faster RCNN can detect objects well with an accuracy value of 72- 99% (Gavrilescu, Zet, Fosalau, Skoczylas, & Cotovanu, 2018) (Cai, Li, Xie, Zhao, & Lu, 2018) (Chandan, Jain, Jain, & Mohana, 2018)(Zhang et al., 2020). Several research methods have been carried out to restore blurred images, including the dark channel method (dark layer) which gives better results compared to other methods with a success rate of around 27.94 dB (decibels) compared to other methods (Pan, Sun, Pfister, & Yang, 2018)(Zhou, Zhuang, Xiong, Zhao, & Du, 2020). In another study conducted by (Anger, Facciolo, & Delbracio, 2019) who used L0 Gradient Before restoring a blurred image. The research carried out gave the results obtained in the form of good performance which can be seen from several tests carried out in the form of various images and the addition of noise. Based on the research reference, the method used in this research for restoring a blurred image is L0 Gradient Prior. The purpose of this research is to increase the accuracy of object detection in the image by adding a pre-processing stage in the form of improving image quality by returning a blurred image so that the object detection results are expected to have better accuracy and be able to recognize more objects in the image. RESEARCH METHODS The research method used is L0 Gradient Prior to restore blurred images and Faster R-CNN for object detection in video images. L0 Gradient Prior method consists of 3 simple stages, namely Multiscale Kernel Estimation, Sharp Prediction, and Kernel Prediction. In figure 1, shows the flowchart of L0 Gradient Prior. Faster R-CNN algorithm is divided into 2 important parts, namely the Regional Proposal Network (RPN) and the Classifier. RPN is used to find the input results in the image that allows the location of the object quickly. The results of the RPN process will later be made in the form of an RoI (Region of Interest). The classifier is a process that classifies RoI from the previous step into corresponding classes (Abbas & Singh, 2018). In figure 2, shows the flowchart of Faster R-CNN. Figure 1 L0 Gradient Prior Flowchart Figure 2 Faster RCNN Flowchart 89 Types of research This research is related to computer visualization and image processing that deals with detecting examples of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos. Time and Place of Research In this study, The dataset is collected from seven intersections in the Danish cities of Aalborg and Viborg. The resolution of cameras are 640x480 pixels and the frame rate is fixed at 20 frames/second. CCTV video data on datasets taken at night were not used because the lighting conditions at night made object detection difficult. Research Target / Subject The purpose of this research is to increase the accuracy of object detection in the image by adding a pre-processing stage in the form of improving image quality to eliminate blurred images, so that object detection results are expected to have better accuracy and be able to recognize more objects. Procedure This type of research focuses on improving the quality of the image which will then proceed to the object detection process stage. Improving the quality of the image is done through the process of returning the blurred image using L0 Gradient Prior. The results of the blurred image return process are then processed to the object detection stage with the Faster R-CNN algorithm. After all, processes are successfully carried out, the stages of testing the results obtained using the Confusion Matrix are carried out. The next step is to compare the results of object detection accuracy before and after improving image quality. It aims to see the level of accuracy in this study. Data, Instruments, and Data Collection Techniques The dataset used in this study is the AAU RainSnow Traffic Surveillance Dataset (Bahnsen & Moeslund, 2018). This dataset contains CCTV video data that monitors road conditions in rainy and snowy conditions. Data analysis technique In this study, we will use a CCTV capture dataset in which there is a blurred image object. Open cv library is used to convert a video into a series of images. Furthermore, the series of images is processed to restore the blurred image using L0 Gradient Prior. Furthermore, the data will be separated into two parts, namely data for training and data for testing. The test data are then analyzed to determine the accuracy of the research results using the confusion matrix. RESULTS AND DISCUSSION Result of L0 Gradient Prior The results of the implementation of blurred image improvement can be seen in Figures 3 and 4. Figure 3 shows a sample image on CCTV video before the implementation of blurred image correction. In Figure 4, after the process of correcting the blurred image using the L0 Gradient Prior algorithm, it looks sharper than Figure 3. Figure 3. Sample image of CCTV Video before the implementation of blurred image correction Figure 4. Sample image of CCTV Video after the implementation of blurred image correction Result of Faster RCNN In this study, The object detection process is divided into two processes, namely the training process and the testing process. In the training process, objects trained on the system are limited to a minimum size of 50 x 50 pixels to reduce object 90 detection errors caused by the size of the trained object being too small. There were 2 (two) CCTV video conditions observed, namely the condition of the CCTV video that had not been processed to restore the blurred image and the condition of the CCTV video that had gone through the process of returning the blurred image. Examples of CCTV video images that have been processed with Faster RCNN can be seen in Figure 5 and Figure 6. Figure 5 shows a sample image on CCTV video before processing to restore the blurred image. Figure 6 shows a sample image on CCTV video after processing to restore the blurred image. The results of object detection on 2 (two) CCTV video conditions can be seen in table 1. In table 1, TP represents the system successfully classifying objects moving vehicles on video correctly, FP represents the system incorrectly recognizing objects that are not vehicles but are recognized as vehicles, TN represents the system successfully correctly detects non-vehicle objects, and FN represents the system failed to detect moving vehicles as non-vehicle objects. Figure 7 shows a sample image on CCTV video after processing to restore the blurred image, the results of object detection with the Faster RCNN algorithm show an error in object detection. Figure 5. Sample object detection image on CCTV video before processing to restore the blurred image Figure 6. Sample object detection image on CCTV video afterprocessing to restore the blurred image Figure 7. Sample image on CCTV video after processing to restore the blurred image that has an error in object detection The results of table 1 then calculate the detection accuracy using the Confusion Matrix to measure the accuracy of the object detection results. The calculation results can be seen in table 2. Based on the data in tables 1 and 2, it can be concluded that the accuracy of object detection decreased slightly when CCTV video was given the process of returning a blurred image on video testing 1 but the number of objects detected by the system increased in test videos 1 and 2. Tabel 1. Object detection result in 2 CCTV video condition Video Condition Number of Object Detected Test Video 1 Test Video 2 TP FP TN FN TP FP TN FN Video without deblur process 394 0 0 0 212 0 0 0 Video with deblur 439 5 0 0 233 0 0 0 91 Tabel 2. Accuracy calculation result using the Confusion Matrix Video Condition Confusion Matrix Result for Accuracy Value (%) Test Video 1 Test Video 2 Video without deblur process 100 100 Video with deblur process 98,87 100 CONCLUSIONS AND SUGGESTIONS Conclusion The accuracy of object detection decreased slightly when CCTV video was given the process of returning a blurred image on video testing 1 but the number of objects detected by the system increased in test videos 1 and 2. This means that by applying the blur image return algorithm, the number of objects that can be recognized is more than without the application of the blur image return algorithm Suggestion In this study, it is still not able to recognize objects with a size of less than 50 x 50 pixels so that in the future it is recommended to apply an algorithm to increase the size of the image so that small objects can still be detected. ACKNOWLEDGEMENT The researcher would like to thank the Ministry of Research and Technology / National Research and Innovation Agency (RistekBrin) who funded this research from start to finish to the publication stage. REFERENCES Abbas, S. M., & Singh, S. N. (2018). Region-based Object Detection and Classification using Faster R-CNN. 2018 4th International Conference on Computational Intelligence & Communication Technology (CICT), 1–6. Ghaziabad: IEEE. https://doi.org/10.1109/CIACT.2018.84804 13 Anger, J., Facciolo, G., & Delbracio, M. (2019). Blind Image Deblurring using the l0 Gradient Prior. IPOL Journal · Image Processing On Line, 9, 124–142. https://doi.org/doi.org/10.5201/ipol.2019.2 43 Bahnsen, C. H., & Moeslund, T. B. (2018, September 21). AAU RainSnow Traffic Surveillance Dataset. Retrieved January 11, 2022, from Kaggle website: https://www.kaggle.com/aalborguniversity/ aau-rainsnow/metadata Borel-Donohue, C. C., & Young, S. S. (2019). Image quality and super resolution effects on object recognition using deep neural networks. Https://Doi.Org/10.1117/12.2518524, 11006, 596–604. Baltimore: SPIE. https://doi.org/10.1117/12.2518524 Cai, W., Li, J., Xie, Z., Zhao, T., & Lu, K. (2018). Street object detection based on faster R-CNN. Chinese Control Conference, CCC, 2018-July, 9500–9503. https://doi.org/10.23919/CHICC.2018.8482 613 Chandan, G., Jain, A., Jain, H., & Mohana. (2018). Real Time Object Detection and Tracking Using Deep Learning and OpenCV. Proceedings of the International Conference on Inventive Research in Computing Applications, ICIRCA 2018, 1305–1308. https://doi.org/10.1109/ICIRCA.2018.85972 66 Gavrilescu, R., Zet, C., Fosalau, C., Skoczylas, M., & Cotovanu, D. (2018). Faster R-CNN:an Approach to Real-Time Object Detection. EPE 2018 - Proceedings of the 2018 10th International Conference and Expositions on Electrical And Power Engineering, 165–168. https://doi.org/10.1109/ICEPE.2018.85597 76 Hasirlioglu, S., Reway, F., Klingenberg, T., Riener, A., & Huber, W. (2019). Raindrops on the windshield: Performance assessment of camera-based object detection. 2019 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2019. https://doi.org/10.1109/ICVES.2019.89063 44 Pan, J., Sun, D., Pfister, H., & Yang, M. H. (2018). Deblurring Images via Dark Channel Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(10), 2315–2328. https://doi.org/10.1109/TPAMI.2017.27538 04 Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R- CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. 92 https://doi.org/10.1109/TPAMI.2016.25770 31 Roy, S. D., & Bhowmik, M. K. (2021). Annotation and Benchmarking of a Video Dataset under Degraded Complex Atmospheric Conditions and Its Visibility Enhancement Analysis for Moving Object Detection. IEEE Transactions on Circuits and Systems for Video Technology, 31(3), 844–862. https://doi.org/10.1109/TCSVT.2020.29911 91 Vidal, R. G., Banerjee, S., Grm, K., Struc, V., & Scheirer, W. J. (2018). UG2: A video benchmark for assessing the impact of image restoration and enhancement on automatic visual recognition. Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, 2018-January, 1597–1606. https://doi.org/10.1109/WACV.2018.00177 Wu, W., Yin, Y., Wang, X., & Xu, D. (2019). Face detection with different scales based on faster R-CNN. IEEE Transactions on Cybernetics, 49(11), 4017–4028. https://doi.org/10.1109/TCYB.2018.285948 2 Zhang, J., Karkee, M., Zhang, Q., Zhang, X., Yaqoob, M., Fu, L., & Wang, S. (2020). Multi-class object detection using faster R-CNN and estimation of shaking locations for automated shake- and-catch apple harvesting. Computers and Electronics in Agriculture, 173, 105384. https://doi.org/10.1016/J.COMPAG.2020.10 5384 Zhou, K., Zhuang, P., Xiong, J., Zhao, J., & Du, M. (2020). Blind Image Deblurring with Joint Extreme Channels and L0-Regularized Intensity and Gradient Priors. Proceedings - International Conference on Image Processing, ICIP, 2020-October, 873–877. https://doi.org/10.1109/ICIP40778.2020.91 91010