Engineering, Technology & Applied Science Research Vol. 9, No. 2, 2019, 3939-3944 3939 www.etasr.com Matlani & Shrivastava: An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid … An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid Feature Selection Techniques Princy Matlani Department of Computer Science & Engineering Guru Ghasidas University Bilaspur, India princy.matlani@gmail.com Manish Shrivastava Department of Computer Science & Engineering Guru Ghasidas University Bilaspur, India manbsp@gmail.com Abstract—As an emerging development in the digital technology era, video processing is useful in a wide range of applications. In the current paper, an algorithm is proposed which is useful for smoke detection in video processing. The algorithm quickly detects fire by eliminating common interruptions like noise, overlapping due to the collision, etc. The proposed algorithm is composed of several techniques such as Haar feature, Bhattacharya distance method, SIFT descriptors, Gabor wavelets approach and SVM classifier to identify the smoke by video processing. Foreground object is identified using a moving object algorithm by predicting the movement of smoke in stable images. The implementation has been carried out in MATLAB. Keywords-smoke detection; Bhattacharya distance; video processing; bounding box technique; SIFT; Gabor wavelet approach; hybrid algorithm; hill climbing algorithm; moving object algorithm I. INTRODUCTION Fire and smoke are needed to get detected at a very early stage in order to minimize the damage they cause. As we know, smoke is considered to be an initial indication of fire, thus, a smoke detecting technique is proposed for early detection and to avoid uncontrolled fires. Conventional point based sensors work by ionization or photometry techniques for detecting the existence of smoke particles whereas, video based smoke detection system uses a newly developed technique based on pattern recognition [1]. Compared to this conventional sensor based technology, video-based fire detection has the advantage of being applicable to large areas, having early response, and long distance of detection. Although researchers have expanded the field of video based smoke detection, most of current methods for video based smoke detection have high rates of false alarms, and low accuracy. The change in transparency of smoke is considered as main feature in [2]. The research is based on edge analysis of background objects, i.e. if edges of background objects are appeared to lose sharpness and become blurred, the object may be considered to be smoke. Despite the significant research done in video-based fire detection algorithms, these methods were found difficult in comparison due to insufficient common evaluation schemes and datasets [3, 4]. Thus, there is a huge need of such a comparison to take place among several fire detection methods to prove their effectiveness. The effectiveness of detection strongly depends on both the distance of the fire source from the point of detection and the dimension of the monitored areas [5]. In this context, several methods have been developed for automatic smoke detection in videos [6, 7]. Primarily, they consider several features of smoke such as its color, motion, and shape. A fire detection system should be designed in such a manner that it can successfully detect smoke with complex shapes [8]. The model proposed in [8] uses dark channel priori based early smoke detection algorithm. A group of pixels is not considered as smoke until they satisfy all considered factors. To achieve an efficient smoke detection in video processing, a hybrid algorithm was proposed. Recursive algorithms are used in real world applications, hybrid algorithms are used especially in the implementation of the divide or conquer algorithms, whereas the size of the data declines as recursion moves deeper [9, 10]. Hybrid algorithms have better results as compared to that of applying individual algorithms because they involve a combination of the desired features of each algorithm. Several methods of automatic smoke detection in video have been developed. But, existing smoke detection methods have high false detection rates. Existing methods require close proximity to the smoke and do not provide information about the type and amount of smoke. It is difficult to exactly detect smoke in video because of reasons such as:(i) smoke color differs largely and several non-smoke substances share the same color with smoke, (ii) smoke textures and shapes often alter erratically, (iii) smoke often blurs images, and causes unreliable features extracted from these blurred images and (iv) foreground objects hide smoke. In this paper, we propose a robust video smoke detection system. This paper has various contributions to video smoke detection. First, we combine extended Haar-like features and statistical measures together, and these features are extracted from both intensity and saturation components of original RGB images. The proposed method then aims to design a novel smoke detection technique employing SIFT feature detection Corresponding author: Princy Matlani Engineering, Technology & Applied Science Research Vol. 9, No. 2, 2019, 3939-3944 3940 www.etasr.com Matlani & Shrivastava: An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid … technique along with SVM classifier in order to decide the correct location of smoke within the image frame as well as the instance of the incident. It also aims to provide a better performance in terms of classification accuracy. II. LITERATURE REVIEW Authors in [11] proposed the h-LDS descriptor, based on the higher order decomposition of the multidimensional image data and enabled the study of dynamic textures using information from various image elements. Compared with standard LDS descriptor, the particle swarm optimization approach is used by a combination of multidimensional texture analysis with the spatiotemporal modeling of smoke to analyze dynamic texture information. In [4], authors proposed a new idea with semi-transparent properties based algorithm of video smoke detection. The marginal area of smoke is semi- transparent. Authors in [2] focused on the verification of detection of smoke in video sequences by spatiotemporal local binary patterns. The following contributions were made: At first, numerous types of 2D and 3D variants in local binary patterns (LBPs) were investigated according to changing properties of smoke, and then, brightness map, Laplacian map and edge map were focused during the study of spatio-temporal LBP. Authors in [8] proposed a tactic based on dark channel priority for early stage smoke detection in video. Smoke indicates proof of fewer chrominance parts and chromaticity examination consumed to distinguish motion behaviors in video arrangements. Motion history image (MHI) strategy is used to distinguish the motion attributes of smoke. In [12], two kinds of histogram are proposed by varying luminance/chrominance texture and its shape. Linear interpolated luminance/chrominance subtraction picture is entered for periodical examination after the threshold position. Authors in [13] propose an efficient smoke detection technique for intelligent video surveillance algorithm having both dynamic and static features of smoke which exhibit contrast, color, texture features, and motion. They are using detection color image segmentation, temporal and spatial wavelet for flame detection. A flame region is also extracted. Authors in [5] proposed a high-speed and early smoke detection method that tracks the local fractal feature of smoke areas based on discrete cosine transform (DCT) coefficients computed at the compression step. Authors in [14] made improvements in Yuan’s video based smoke detection and used made experiments of the accumulative motion orientation. Optimal thresholds for motion, chrominance recognition and noisy blocks were eradicated. The experiment in [15] describes an adaptive LOG algorithm for smoke detection by energy difference. Once the smoke region is identified, dynamic accumulation, separating and extricating smoke leakage position would be accomplished. Authors in [16] made the utilization of optimal mass transport (OMT) optical flow. The process of detection is encapsulated as an administered Bayesian classification. Feature vectors are embodied of OMT velocities and red, green, blue color channels. The classifier is used as a specific hidden layer neural network. Authors in [17] proposed a technique, composed of the subsequent basic steps: The initial step is selecting whether the camera is stationary or dynamic. The next step is finding the regions of motion occurred in the present input frame over the background frame of the image. And finally, regions of interest (ROIs) are pointed out through associated in-component testing. The block-based approach should be enforced for the first two steps. In the last step, the patterned ROI is confirmed as smoke by utilizing the k-temporal representation of its shape and color separated from the ROI. Authors in [18] proposed turbulent smoke phenomena over the relationship of area and perimeter. Authors in [19] gave a target-tracking established fire/smoke detection technique for an early fire alarming system in broad or wide spaces. Brightness consistency, motion accumulation, and spread were effectively studied through improved Gaussian mixture model positioning algorithm. It also gave three efficient static and dynamic visual features of smoke. Authors in [20] investigated the approach of smoke pixel judgment made out of two decision principles: chromaticity-based static decision rule and diffusion-based dynamic characteristic decision rule. The chromatic rule assumes that the smoke has grayish color. The scattering properties of smoke were used as dynamic characteristics. III. RESEARCH METHODOLOGY The flow chart of the proposed smoke detection method is shown in Figure 1. Block based implementation is used to ensure efficiency and robustness. Flowchart of the proposed method. A. Converingt Video into Frames The given input video has been converted into 693 frames using a standard MATLAB code. In our code, for converting the video frames, many functions were used. For example, Video Reader class was used to determine the starting time of the video. Then we used read() function to extract a frame from the video at an exact particular time. Also, imwrite function Engineering, Technology & Applied Science Research Vol. 9, No. 2, 2019, 3939-3944 3941 www.etasr.com Matlani & Shrivastava: An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid … was used to store the images in a folder. In this manner, 693 frames have been converted from the given input video. B. Background Subtraction The converted video frames were further processed by background subtraction methods for object localization. Moving pixels in the smoke image are calculated by deducting the current image of background image based on the threshold value. Background subtraction, also termed as foreground detection, is a procedure in which the center of the image is extricated for further processing. Background subtraction is an extensively utilized process to detect moving objects in a video captured on a static camera. The basic idea in this approach is to identify the moving objects by the change that occurs in the present frame and a reference frame referred as “background image”. The reference background and threshold is dynamically updated with incoming frames. Smoke detected output after background subtraction is shown in Figure 2. According to background subtraction method, the background frame is set as: , when current frame is the first frame (((1 ) ( )), otherwise   =   −  +   bframe bframe r tframe r bframe where bframe is the background frame, tframe is the current frame and r is a real and positive constant close to one. Moving objects detected using background subtraction (white: no smoke, black: smoke) The algorithmic steps are: • Select the background frame and current frame from the given video. • Convert into grayscale those selected frames. • For calculation of pixel value, assume the frame dimension. • Find the difference in pixels of the selected two frames which should be match with a defined threshold value. • Update the threshold value according to the changes in frames. The algorithm is detecting the moving regions from deducting the current image pixel-by-pixel from a reference background image which is poised by averaging image. C. Bounding Box Technique In the bounding box technique, the major work is to read the image and convert it into a gray image. Further, it changes to a binary image using certain techniques. After converting the given image into a binary image, it is necessary to fill the acquired holes in the image. This work is done by using the bounding box technique. The region properties of the image have been considered in a regular manner for improving the performance of the algorithm. The bounding box will provide the properties of the given frame and the minimum or least bounding or enclose box for a point set (S) in N dimensions. This is the exact box with the fundamental measure (area, volume or hyper volume in greater dimensions) within which the entire points lie (Figure 3). Bounding box output D. Bhattacharya Distance Technique We prepared the averaged histogram HS and HN from manually selected smoke and non-smoke image blocks. The Bhattacharyya coefficient represents the similarity of two histograms from 0 to 1. The Bhattacharya distance measure is mainly used for comparing the similarity among two histograms for the background subtracted resultant image as follows: Let Ri be the frequency coded quantity for the first histogram and Si be the similar quantity for the second histogram. For probability distributions m and n over the particular domain i, the Bhattacharya distance is described as: (1) where, ( , ) ( ) ( ) i I BC m n m i n i  =  (2) In (1)-(2), BC is the Bhattacharya coefficient for distance probability distribution. For continuous probability distributions, the Bhattacharya coefficient has been defined as: ( , ) ( ) ( )BC m n m i n i di=  (3) ( , ) ln( ( , )) B D m n BC m n= − Engineering, Technology & Applied Science Research Vol. 9, No. 2, 2019, 3939-3944 3942 www.etasr.com Matlani & Shrivastava: An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid … In either case, 0 1 BC and 0    B D . DB doesn't adhere to the triangle inequality, yet the Hellinger distance 1− BC is observed to the triangle inequality. The Bhattacharya distance is applied to all frames and the result is saved. In our research, Bhattacharya distance will be calculated after the video-to- frame conversion and background subtraction. The similarity between two consecutive frames and distance variation is computed by Bhattacharya algorithm. Then the SIFT technique will be applied to calculate the features of Bhattacharya distance. E. SIFT Method SIFT (scale-invariant feature transform) method is applied to extract the features of the smoke obtained from the Bhattacharya distance method. The implementation of SIFT technique contains two steps. The first step extracts the characteristics of a smoke image and then determines its descriptors, which include detection of the characteristics that are most likely to represent smoke. The second step performs the mapping procedure from the obtained results. The features obtained by this method are called as its key points. In our work, we quantized the extracted features and combined the descriptors of the objects which belong to the same category. The relating k-means on these particular features were calculated. The features of k-means were applied to all possible categories for getting extra memory space in MATLAB implementation. The computed values of SIFT descriptors for each image are obtained using the bounding box method. After the similarity distance calculation, SIFT was applied. This technique is employed for locating the features of a particular object in an image consisting of many different objects. There are various examinations in methods of acquiring these features and how to conduct them. SIFT image features produce a set of local features of an object that are not affected by operations like object scaling and rotation. For image feature extraction method, it takes an image as input and converts it into a “large group of local feature vectors”. All these vectors remain constant for any rotation, scaling or image translation. SIFT process includes the following steps: • Scale-space extremes detection. • Localization of key point. • Coordination assignment. • Key point descriptor. SIFT analysis is combined with detecting salient locations in a given image and extracting descriptors that are distinctive yet invariant to changes in viewpoint, illumination, etc. Both standard SIFT point detector and histogram-of-gradients descriptors can be used. The location of the features can be extracted by SIFT. By using SIFT, each key point descriptor extracted from the object image is matched independently to the selected database of descriptors extracted from all collected training images. It will find the best match from each descriptor from all training images. A subsequent threshold will be used for rejecting the key points, if there are not any good matches. Some of the possible matches are: • The whole of an image matches with the whole of another selected image. • A single part of an image matches with the whole of another image. • One image part matches with a part of another image. F. Gabor Wavelet Approach Gabor wavelet transform analyzes the local structure based on spatial frequency, spatial localization and orientation selectivity for the extracted feature of the image from SIFT. Among various wavelets, the Gabor function provides the optimal resolution for the extracted smoke region in both time and frequency domain. The computation process involves two variables: frequency and orientations. To evaluate the magnitude of local frequencies of that estimated wavelength and coordination in the image, it is then convolved with the image. The Gabor wavelets are measured as a class of self- similar functions. The equation of Gabor wavelet is: (4) 1 4     t f Among all types of functions, the Gabor function is found to attain the minor bound and it also executes the finest analytical resolution in the joint domain. The characteristics of image texture can be obtained by convolving the image by Gabor filters. Gabor filters with different frequencies and orientations are selected to obtain the texture features of the smoke regions. In order to optimally sense and localize features at various scales, filters with varying parameters are used rather than a fixed one. This approach is used for certain applications like Gabor networks for face reconstruction, facial expression classification, iris recognition, and fingerprint recognition. Gabor features are approximated by using many gray values. The region is equivalent to the object which is usually segmented correctly in all images. The different set of Gabor features is typically related with the segment containing the object of interest. IV. RESULTS AND DISCUSSION In our research, we focused on detecting smoke in video processing. Hence, moving object algorithm is developed to predict moving objects. In a video stream, the objects acquire a stability phase while the others remain the same. Hence, it has an ability to detect moving objects and they are represented as smoke. This result has been implemented in MATLAB. In our experiments, initially we considered two videos prerecorded from YouTube.com for smoke detection (Figures 4-5 and 6-8). Our program is implemented using MATLAB 2015. Figure 9 shows the hybrid algorithm technique which degrades the clarity and reality of the original video. Hence, it should be predicted using a suitable technique. Figure 8 shows an exact output regarding smoke detection of the previous images. 2222 2 2 2 2 ( )( ) , ( ) ( ) t f f p f dft p t dt p t dt p f df    = =    Engineering, Technology & Applied Science Research Vol. 9, No. 2, 2019, 3939-3944 3943 www.etasr.com Matlani & Shrivastava: An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid … Blob detection of smoke in video1 taken at t1 Blob detection of smoke in video1 taken at t2 Blob detection of smoke in video2 at time t1 Blob detection of smoke in video2 at t2 Input video of smoke detection process with detected smoke area Output image of predicted smoke Table I displays a comparison of the proposed system with [21] and [22]. Three smoke containing videos and three non- smoke videos were considered and compared: 1) black smoke from a standing car, 2) white smoke from a vehicle, 3) black smoke from a car. The proposed method is detecting the smoke’s exact shape in less time. With the proposed method, the time consumption for the overall setup was minimized. TABLE I. PERFORMANCE ANALYSIS Videos Movie 1 Movie 2 Movie 3 Time/Frame (msec) 117.334462/187 64.304860/142 /243 Alarm at frame number Proposed 91 89 82 [21] 164 216 68 [22] 89 94 99 Description Black smoke from standing car White smoke from vehicle Black smoke from car V. CONCLUSION This paper proposes a new smoke recognition system. Many techniques and algorithms have been used for resolving issues like noise reduction, reliability, and cost-effectiveness. This research concentrates on the detection of: transparent smoke, dense smoke and non-smoke regions using various approaches like bounding box technique for the comparison of different frames, Bhattacharya distance for the comparison of Engineering, Technology & Applied Science Research Vol. 9, No. 2, 2019, 3939-3944 3944 www.etasr.com Matlani & Shrivastava: An Efficient Algorithm Proposed For Smoke Detection in Video Using Hybrid … similarities between two histograms, SIFT technique for the extraction of original images from different frames, and Gabor wavelet approach for the application of smoke detection approach. Finally, moving object algorithm has been used for the detection of smoke as the only moving object. The experimental results show that the discussed method can be used for the accurate detection of smoke at a very fine rate. The execution has been carried out in MATLAB. In future, open research problems should be analyzed by more advanced techniques. REFERENCES [1] H. Tian, W. Li, L. Wang, P. Ogunbona, “A Novel Video-Based Smoke Detection Method Using Image Separation”, IEEE International Conference on Multimedia and Expo, Melbourne, Australia, July 9-13, 2012 [2] M. Favorskaya, A. Pyataeva, A. Popov, “Verification of Smoke Detection in Video Sequences Based on Spatio-temporal Local Binary Patterns”, Procedia Computer Science, Vol. 60, pp. 671-680, 2015 [3] C. R. Steffens, R. N. Rodrigues S. S. D. C. Botelho, “An Unconstrained Dataset for Non-Stationary Video Based Fire Detection”, 12th Latin American Robotics Symposium and 3rd Brazilian Symposium on Robotics (LARS-SBR), Uberlandia, Brazil, October 29-31, 2015 [4] Y. De-Fei, H. Ying, B. Feng-Long, “Video smoke detection based on semitransparent properties”, The 27th Chinese Control and Decision Conference, Qingdao, China, May 23-25, 2015 [5] A. Benazza-Benyahia, N. Hamouda, F. Tlili, S. Ouerghi, “Early smoke detection in forest areas from DCT based compressed video”, 20th European Signal Processing Conference (EUSIPCO), Bucharest, Romania, August 27-31, 2012 [6] R. D. Labati, A. Genovese, V. Piuri, F. Scotti, “Wildfire Smoke Detection Using Computational Intelligence Techniques Enhanced With Synthetic Smoke Plume Generation”, IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 43, No. 4, pp. 1003-1012, 2013 [7] C. Y. Lee, C. T. Lin, C. T. Hong, M. T. Su, “Smoke detection using spatial and temporal analyses”, International Journal of Innovative Computing, Information and Control, Vol. 8, No. 7, pp. 4749-4770, 2012 [8] L. Miao, Y. Chen, A. Wang, “Video smoke detection algorithm using dark channel priori”, 33rd Chinese Control Conference, Nanjing, China, July 28-30, 2014 [9] S. M. Ma, Y. Sun, A. P. Li, “A simulated annealing algorithm for multi- objective hybrid flow shop scheduling”, in: Design, Manufacturing and Mechatronics, pp. 1463-1473, World Scientific, 2015 [10] T. Ohashi, Z. Aghbari, A. Makinouchi, “Hill-climbing algorithm for efficient color-based image segmentation”, IASTED International Conference on Signal Processing, Pattern Recognition, and Applications, June 30-July 2, Rhodes, Greece, 2003 [11] K. Dimitropoulos, P. Barmpoutis, N. Grammalidis, “Higher Order Linear Dynamical Systems for Smoke Detection in Video Surveillance Applications”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, No. 5, pp. 1143-1154, 2017 [12] I. F. Ince, G. Y. Kim, G. H. Lee, J. S. Park, “Patch-wise periodical correlation analysis of histograms for real-time video smoke detection”, 2014 IEEE International Conference in Industrial Technology, Busan, South Korea, February 26-March 1, 2014 [13] R. Bohush, N. Brouka, “Smoke and flame detection in video sequences based on static and dynamic features”, Signal Processing: Algorithms, Architectures, Arrangements, and Applications, Poznan, Poland, September 26-28, 2013 [14] O. B. Alejandro, M. G. Leonardo, S. P. Gabriel, T. M. Karina, N. M. Mariko, “Improvement of a Video Smoke Detection Based on Accumulative Motion Orientation Model”, IEEE Electronics, Robotics and Automotive Mechanics Conference, Cuernavaca, Mexico, November 15-18, 2011 [15] Y. Liu, G. Liu, “A smoke detection algorithm of energy difference between frames based on adaptive LOG operator on the infrared video processing”, Second International Conference on Mechanic Automation and Control Engineering, Hohhot, China, July 15-17, 2011 [16] I. Kolesov, P. Karasev, A. Tannenbaum, E. Haber, “Fire and smoke detection in video with optimal mass transport based optical flow and neural networks”, 2010 IEEE International Conference on Image Processing, Hong Kong, China, September 26-29, 2010 [17] D. Kim, Y. F. Wang, “Smoke detection in video”, 2009 Congress on Computer Science and Information Engineering, Los Angeles, USA, March31-April 2, 2009 [18] C. Ho, T. Kuo, “Real-time video-based fire smoke detection system”, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Singapore, July 14-17, 2009 [19] Z. Wei, X. Wang, W. An, J. Che, “Target-tracking based early fire smoke detection in video”, Fifth International Conference on Image and Graphics, Xi'an, China, September 20-23, 2009 [20] T. H. Chen, Y. H. Yin, S. F. Huang, Y. T. Ye, “The smoke detection for early fire-alarming system base on video processing”, 2006 International Conference on Intelligent Information Hiding and Multimedia, Pasadena, USA, December 18-20, 2006 [21] B. U. Toreyin, Y. Dedeoglu, U. Gudukbay, A. E. Cetin, “Computer vision based method for real-time fire and flame detection”, Pattern Recognition Letters, Vol. 27, No. 1, pp. 49-58, 2006 [22] F. Yuan, “Video-based smoke detection with histogram sequence of LBP and LBPV pyramids”, Fire Safety Journal, Vol. 46, No. 3, pp. 132- 139, 2011 AUTHORS PROFILE Princy Matlani (Orchid id: https://orcid.org/0000-0002- 7421-8366), was born in Bilaspur, India, in 1990. She received the BE Degree in Computer Science & Engineering from Government Engineering College, Bilaspur, India, in 2011, and the MTech. degree from the National Institute of Technology (NIT), Raipur, India, in 2013. She is an active member of IEEE, and pursuing her PhD from Guru Ghasidas University, Bilaspur, India. She is experienced in working in image processing field, and currently works on computer vision based surveillance systems. Dr. Manish Shrivastava (Orchid id: https://orcid.org/0000-0002-1346-968X), Assistant Professor, Department of Computer Science & Engineering, Institute of Technology, Guru Ghasidas University, Bilaspur. He obtained his MTech Degree from DAVV, Indore, and PhD from Guru Ghasidas University, Bilaspur. He has about fifteen years of teaching and research experience, and a number of papers in various national and international journals to his credit. His field of interest is network security and image processing. A life member of the Indian Society for Technical Education, Sr. member of IACSIT, and a life member of the Cryptology Research Society of India. https://orcid.org/0000-0002-7421-8366 https://orcid.org/0000-0002-7421-8366 https://orcid.org/0000-0002-1346-968X