Microsoft Word - cet-01.docx CHEMICAL ENGINEERINGTRANSACTIONS VOL. 46, 2015 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors:Peiyu Ren, Yancang Li, Huiping Song Copyright © 2015, AIDIC ServiziS.r.l., ISBN978-88-95608-37-2; ISSN 2283-9216 Multifocus Image Fusion Based on Multiwavelet and DFB Changhong Chen a, Peng Geng*b, Kai Lub a School of Mechanical Engineering, Shijiazhuang Tiedao University, Shijiazhuang, 050043, China b School of Information Science and Technology, Shijiazhuang Tiedao University, Shijiazhuang, 050043, China GengPeng@stdu.edu.cn Aiming at solving the fusion problem of multifocus, a novel image fusion algorithm is presented in this paper. Firstly, the multiwavelet is combined with the direction filter bank (DFB) to construct the proposed MDFB transform to overcome defect of the contourlet transform. Secondly, the source multifocus images are decomposed into lowpass coefficients and highpass coefficients. Thirdly, the local sum-of-laplacian are adopted to select the coefficient of clear image pixel. Finally, the inverse MDFB transform are performed on the fused coefficient to reconstruct the merged image. The experiments demonstrate that the presented fusion method is more effective than the contourlet-based fusion method and NSCT-based methods in terms of both visual quality and objective evaluation. 1. Introduction Image fusion is the combination of two or more different images to form a new image by using a certain algorithm (Kavitha and Chellamuthu (2014)). The combination of sensory data from multiple sensors can provide more reliable and accurate information. It forms a rapidly developing research area in remote sensing, medical image processing, and computer vision (Duan, Meng and Xiang et al.(2015)). Generally, the fusion algorithm can be classified into two categories. One is the spatial domain-based methods, which directly select pixels, blocks or regions from clear parts in the spatial domain to compose fused images (Di, Jingwen and Xiaobo (2015). Another class is merged the coefficients of multiscale transform domain by different fusion rule (Yang, Guo and Ni (2008). In the second class, the wavelet, curvelet, contourlet, bandlet are introduced to merge the source images. Contourlet transform has better performance in representing the image salient features such as edges, lines, curves and contours than wavelet transform because of its anisotropy and directionality (Piella(2003)), (Yipeng, Jing and Qiang et al. (2005)). Regional variance and local energy are adopted as fusion rules to merge the lowpass and high-pass subbands of contourlet, respectively (Yang, Guo and Ni (2007). Sum-modified-Laplacian are introduce into fused the contourlet coefficients instead of spatial domain (Qu, Yan and Yang (2009)). Besides, some combination methods were proposed to fusion different types of images. Li combined the traditional wavelet and the curvelet to fused the multifocus images (Li and Yang (2008)). Zou proposed a multifocus image fusion method by directly combining the Multiwavelet and Contourlet (Zou, Guo and Tian (2012)). A novel remote sensing image fusion is proposed by combining the multiwavelet transform with high-pass filter (Yan, Shen and Wu et al.. (2007)). The contourlet transform consists of two steps which are the subband decomposition and the directional transform. In comparison with the atrous wavelet transform used as the pyramid decomposition in contourlet (Do and Vetterli (2005)), the multiwavelet is orthogonal and symmetric and more redundancy but possesses the property of compact support. Therefore, we propose a new multi-resolution and multi-scale image representation method named as MDFB transform that the multiwavelet transform is combined with direction filter bank (DFB) in this paper. The MDFB transform is more effective to capturing the detailed information in source multifocus images than contourlet transform. Therefore, the fuse result can overpass the contourlet-based fusion method. DOI: 10.3303/CET1546047 Please cite this article as: Chen C.H., Geng P., Lu K., 2015, Multifocus image fusion based on multiwavelet and dfb, Chemical Engineering Transactions, 46, 277-282 DOI:10.3303/CET1546047 277 2. MDFB transform 2.1 Multiwavelet Donovan applies the fractal interpolation approach to reconstruct the Geronimo, Hardin and Massopust (GHM) multiwavelet (Zhang, ZhiJun and Shengqian et al. (2009)) which support basis is in [0 2] [16]. Multiwavelet is orthogonal, symmetric, high approximation and good regularity. Both the multiwavelet and the scalar wavelet are based on multiscale geometry analysis theory. Multiwavelet is composed of the scale function T 1 2 r(t) [ (t), (t),... (t)]φ φ φΦ = and the wavelet function T 1 2 r(t) [ (t), (t),... (t)]ψ ψ ψΨ = after translation and expansion. The multiwavelet two-scale equations verified the following: 0 ( ) 2 (2 ) Z L k k t H t k k = Φ = Φ − ∈ (1) 0 ( ) 2 (2 ) Z L k k t G t k k = Ψ = Ψ − ∈ (2) Where l is the number of scaling coefficients and kH and kG are the lowpass and highpass matrix filter for each translation distance k , respectively. There are r (r = 2) scaling function in the multiwavelet transform. 2.2 The MDFB Transform Implement The contourlet transform consists of two steps which is the subband decomposition and the directional transform. A Laplacian pyramid is firstly used to capture point discontinuities, then followed by directional filter banks (DFB) to link point discontinuity into lineal structure. In comparison with the atrous wavelet transform used as the pyramid decomposition, the multiwavelet is orthogonal and symmetric and more redundancy but possesses the property of compact support. Therefore, we propose a new image multi-resolution and multi- scale representation method named as MDFB transform that the multiwavelet transform is combined with directional filter banks. In MDFB transform, the image is decomposed into a low-pass LL subband and three high-pass subbands such as LH, HH and HL by the multiwavelet transform instead of Laplacian transform. LH, HH and HL represent the horizontal, vertical and diagonal subband. The Multiwavlet is same as the traditional wavelet transform. The difference is that the size of the image same as the every Multiwavlet subband coefficients instead of the two times as traditional wavelet subband coefficients. During to the more reductancy of Multiwavelet than traditional wavelet transform, the Multiwavlet can more concisely express the images information. After Multiwavelet decomposition, the directional filter banks is subsequently performed to decompose the every subband into several direction coefficients which can capture the image edge and detail information by different direction.Therefore, the MDFB transform is a multi-resolution, multidirectional, multi- scale and anisotropic expression for effectively capturing the detailed information in source multifocus images. In this paper, the two levels decomposition of the multiwavelet is used. 3. Fusion rule Fusion rule is vital in the image fusion methods based on mutiscale geometry analysis theory because different fusion rules in transform domain can produce different quality of fused image. For multifocus image fusion, many typical focus measurements, e.g. variance, energy of image gradient (EOG), tenengrad, spatial frequency (SF), energy of image Laplacian (EOL), and sum-modified-Laplacian (SML) are adopted in literature(Li, Chai and Yin et al. (2012). They all measure the variation of pixels. Pixel with greater values of these measurements, when source images are compared with each other, are considered from the focus regions and selected as the pixels of the fused image. In this paper, the local sum of modified Laplacian (LSML) with weighted SML is adopted as the clarity measure to distinct the MDFB coefficients in clear image part of source images with MDFB coefficients in blurred image part of source images. The complete expression of modified Laplacian (ML) and LSML are shown in Eqs. (3) and (4), respectively. The modified Laplacian (ML) adopts the absolute values of the second derivatives in the Laplacian to avoid the cancellation of second derivatives in the horizontal and vertical directions that have opposite signs. The ML is calculated as follows: ),(),(),(2),(),(),(2),( stepnmIstepnmInmInstepmInstepmInmInmML +−−−++−−−= (3) Where ),( nmI is the located at m-th row and n-th column in an image. The LSML, as the focus measure of image, can be calculated as following in a window around the center point when ‘step’ is set to 1 in the Eq. (3). 278 [ ] ++= a b bnamMLbaWnmLSML 2),(),(),( (4) Where the weighted template W is a template whose size is relatively small, and must satisfy the normalization rule  = 1),( baW . In the traditional sum-of-Laplacian method, all values in W are set to 1. That is to say, all the ML values around the center pixel produce the same effectiveness. In fact, different elements around the window should be different weights because the location relation among the window. Therefore, in order to highlight the center pixel of the window, W is set as:           = 16/18/116/1 8/14/18/1 16/18/116/1 W (5) For the high coefficients and low frequency coefficients, the SML is adopted to select all of coefficient in MDFB domain. 4. Proposed fusion method 1).The proposed MDFB transform are adopted to decompose the source images, respectively. 2).Compute the ( , )ALSML m n and ( , )BLSML m n of the all every subbands according to Eq. (5), separately. 3).Compute the decision map ( , )D m n to select the different MDFB coefficients. The coefficients can be fused by Eq. (7).     < >= = ),(),(,0 ),(),(,1 ),( nmLSMLnmLSMLif nmLSMLnmLSMLif nmD BA BA (6)     = = = 0),(),,( 1),(),,( ),( nmDifnmMDFB nmDifnmMDFB nmMDFB B A F (7) The ),( nmMDFBF , ),( nmMDFBA and ),( nmMDFBB are the coefficient of the fused images F, source image A and source image B located at the m-th row and n-th column of MDFB coefficient, respectively. 4).Finally, the inverse MDFB transform are adopted to reconstruct the fused image by ),(, nmMDFB hlF . The schematic diagram of proposed fusion algorithm can be clearly understood in Fig. 1. Figure 1: Schematic diagram of proposed fusion algorithm. 5. Experiments and analysis To evaluate the performance of the proposed fusion rule, several experiments have been performed on the source images shown in Fig. 2. For comparison purposes, in this paper, the comparison is performed on the contourlet method, nonsubsampled contourlet method, kumar’s method (Kumar (2013)) and proposed MDFB transform method. In the contourlet method, two levels of decomposition are conducted on the source images by ‘db6’ wavelet filters. For the contourlet-based method, the 'pkva' filter and 'cd' filter (Zhan, Teng , and Li et al. (2015)) are adopted in pyramidal filter and directional filter, respectively. For the NSCT-based method, the 'pyrexc' filter and 'pkva' filter are adopted in pyramidal filter and directional filter, respectively. The 2,3,4,4 directions decomposition are used in the level of 1,2,3,4 in the contourlet and NSCT tranform, respectively. The LSML fusion rule are used in the three fusion methods, above. The fused images with the different method are demonstrated in the Fig. 3, Fig. 4, Fig. 5 and Fig. 6. From the fusion results of the three methods, it is easy to find that the fused image with all the methods contains both the pair of source images. However, 279 some difference can be clearly seen by careful observation in Fig.3, Fig. 4, Fig. 5 and Fig. 6. It can be clearly concluded that fused images by proposed method is clearer than the other images fused by the other two methods. Furthermore, there is a highest contrast in the image fused by proposed method than in those by the other two methods. (a) Focus on left (c) Focus on left (e) Focus on left (g) Focus on left (b) Focus on right (d) Focus on right (f) Focus on right (h) Focus on right Figure 2: Source multifocus images in the experiments. (a) NSCT (b)Contourlet (c) Kumar (d) Proposed (e) NSCT (f) Contourlet (g) Kumar (h) Proposed Figure 3: Fusion results and difference images by different algorithms. (a) NSCT (b)Contourlet (c) Kumar (d) Proposed (e) NSCT (f) Contourlet (g) Kumar (h) Proposed Figure 4: Fusion results and difference images by different algorithms. On the other side, the difference images between the fused images with source multifocus image are shown in the second row of Fig. 3, Fig. 4, Fig. 5 and Fig. 6 for clearly illustrating the distinction among the proposed method and other three methods. According to the basic image fusion theory, if we subtract the one piece of source image with the fused image, the residual image will be obtained. The residual image corresponding to the clear part of the source image should be close to zero. Therefore, the less residual information is, the better fused method is. In Fig. 3(e)-(h), It can be seen that there are more residual information in right part of Fig. 3(f) and Fig. 3(g) fused by contourlet method and Kumar’s method than in the other two difference image. 280 It is difficult to find the residual information in right part of Fig.3(g) fused by the presented method. Similarly, the same conclusion can be drawn from the Fig.4 (e)-(h). The Fig. 5 (e)-(h) demonstrated the difference image between Fig. 5 (a)-(d) with Fig. 2(e). The Fig.6 (e)-(h) show the difference image between Fig. 6(a)-(d) with Fig. 2(g). From Fig.5 (e)-(h) and Fig.6 (e)-(h) ,it can be concluded that there is minimum diffidence image with propose method among the residual image with the four methods. (a) NSCT (b) Contourlet (c) Kumar (d) Proposed (e) NSCT (f) Contourlet (g) Kumar (h) Proposed Figure 5: Fusion results and difference images by different algorithms. (a) NSCT (b) Contourlet (c) Kumar (d) Proposed (e) NSCT (f) (g) Kumar (h) Proposed Figure 6: Fused results and difference images by different algorithms. Finally, Tab. 1 shows the objective performance by the MI and QAB/F criteria (Lei, Bin, and Lian-fang (2014)). The higher MI and QAB/F are, the better the fusion result is (Yang and Li (2012)). To sum up, the MI and QAB/F of proposed method are higher than the other method except some special cases. Generally speaking, the presented method can more effectively combine the clear part into the merged image and introduce less artifact than the other methods from the analysis of fused images, difference images and objective criteria. Table 1: Objective criteria comparison on different methods. Image Criteria Contourlet NSCT Kumar Proposed Book MI 6.7843 8.1570 6.3534 8.6108 QAB/F 0.6999 0.7213 0.6566 0.7199 Lab MI 6.7664 7.5750 7.4774 8.1029 QAB/F 0.7026 0.7341 0.7321 0.7384 Disk MI 5.7876 6.7442 6.6735 7.6660 QAB/F 0.6601 0.7027 0.6952 0.7129 Pepsi MI 6.8307 7.3133 7.2282 7.4248 QAB/F 0.7385 0.7746 0.7867 0.7548 281 6. Conclusions In order to improve the effect of multifocus image fusion, an image fusion algorithm is presented in this paper. The multiwavelet is combined with the DFB to construct the proposed MDFB transform. The proposed MDFB transform is not only a 2D image sparse representation method but also a kind of better approximation of image edge. Furthermore, The MDFB transform has the characteristic of multi-scale, multi-direction and anisotropy. The experiments of the images fusion indicate that the suggested fusion scheme is more effective than other image fusion works. Acknowledgments Some of the images adopted in these experiments are downloaded from the website of http: //www.imagefusion.org. This research was partially sponsored by the Natural Science Fund of Hebei Province under grant F2013210094 and F2013210109. References Do M.N., Vetterli M., 2005, The contourlet transform: An efficient directional multiresolution image representation, IEEE Transactions on Image Processing, 12, 2091-2106. Duan J.Y., Meng G.F., Xiang S.M., Pan C.H., 2014, Multifocus image fusion via focus segmentation and region reconstruction, NEUROCOMPUTING, 140 (22), 193-209. Guo D., Yan J.W., Qu X.B., 2015, High quality multi-focus image fusion using self-similarity and depth information, Optics Communications, 338(1), 138-144. Kavitha C.T., Chellamuthu C., 2014, Medical image fusion based on hybrid intelligence.Applied Soft Computing, 20, 83-94. Li H.F., Chai Y., Yin H.P., Guo Q., 2012, Multifocus image fusion and denoising scheme based on homogeneity similarity, Optics Communications, 285(2), 91-100 Li S.T., Yang B., 2008, Multifocus image fusion by combining curvelet and wavelet transform Pattern Recognition Letters, 29(9), 1295-1301. Liu Y.P., Jin J., Wang Q., Shen Y., Dong X.Q., 2014, Region level based multi-focus image fusion using quaternion wavelet and normalized cut, Signal Processing, 97(1), 9-30. Meng Y., Shen X.Y., Wu R.B., Ling H.F., 2007, Remote sensing image fusion using multiwavelet transform combined with HPF+, Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, 1651-1654. Piella G., 2003, A general framework for multiresolution image fusion: from pixels to regions, Information Fusion, 4(4), 259-280. Qu X.B., Yan J.W., Yang G.D., 2009, Multifocus image fusion method of sharp frequency localized Contourlet transform domain based on sum-modified-Laplacian, Optics and Precision Engineering, 17(5):1203-1212. Shreyamsha Kumar B.K., 2013, Image fusion based on pixel significance using cross bilateral filter, Signal, image and video processing, 1-12, DOI: 10.1007/s11760-013-0556-9. Wang L., Li B., Tian L.F., 2014, Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients, Information Fusion, 19, 20-28. Yang B., Li S.T., 2012, Pixel-level image fusion with simultaneous orthogonal matching pursuit, Information Fusion, 13(1), 10-19. Yang L., Guo B.L., Ni W., 2008, Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform, Neurocomputing, 72(3) 203-211. Yang L., Guo B.L., Ni W., 2007, Multifocus image fusion algorithm based on region statistics in Contourlet domain, Journal of Xi'an Jiaotong University, 41(4), 448-452. Zhan K., Teng J.C., Li Q.Q., Shi J.H., 2015, A novel explicit multi-focus image fusion method, Journal of Information Hiding and Multimedia Signal Processing, 6(3), 600-612. Zhang L., Fang Z.J., Wang S.Q., Yang F., Liu G.D., 2009, Multiwavelet adaptive denoising method based on genetic algorithm, Journal of Infrared and Millimeter Waves, 28(3), 77-80. Zou Y.L., Guo Y., Tian L., 2012, Multifocus image fusion combined multiwavelet with contourlet, Research Journal of Applied Sciences, Engineering and Technology, 4(20): 4066-4071. 282