36 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 Satellite Images Fusion Using Mapped Wavelet Transform Through PCA Hameed Majeed Abduljabbar Dept. of Phyiscs/College of Education for Pure Sciences (Ibn Al –Haitham) University of Baghdad Received in :10/May/2016,Accepted in :30/August/2016 Abstract In this paper a new fusion method is proposed to fuse multiple satellite images that are acquired through different electromagnetic spectrum ranges to produce a single gray scale image. The proposed method based on desecrate wavelet transform using pyramid and packet bases, the fusion process preformed using two different fusion rules, where the low frequency part is remapped through the use of PCA analysis basing on covariance matrix and correlation matrix, and the high frequency part is fused using different fusion rules (adding, selecting the higher, replacement), then the restored image is obtained by applying the inverse desecrate wavelet transform. The experimental results show the validity of the proposed fusion method to fuse such images with equally representation comparing with the general wavelet fusion method that fuses the high frequency parts only. Key Words: Fusion, Wavelet Transform, PCA, Pyramid, Packet. 37 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 Introduction Image fusion of different images is acquired from different sensors or modalities to produce single fused image are primarily used for: (a) increases the riability of vision or interpretation process; or (b) as preprocessing step for further processing by different image processing techniques. [1] Designing an effective fusing method is a difficult task since the optimality of fusing images depend on the local relationship among the sensors, therefore, fusing certain image into single image with high signal-to-noise ratio dose not necessary give the same quality using other different images, beside that the attempt to produce a single image is usually combined with the loss of information from the fused images. [2] The most common fusion methods are the wavelet transform and principal component analysis transform, for their ability to separate the high frequency part from the low frequency part. Wavelet decomposition usually preform using two schemes, the first scheme uses a pyramid decomposition in which at each level the decomposition is applied only at the low- low part only. [3] While for more efficient decomposition, the packet scheme is preformed where the decomposition applied onto all parts, the high frequency and low frequency parts at each level recursively [4]. For the first decomposition there is no difference between the pyramid and packet wavelet transform where the same four parts will appear ( HHHLLHLL IIII ,,, ) as the decomposition result, the packet procedure will take effect starting from the second decomposition level where instead of the seven parts with only one low frequency part, the packet transform will produce sixteen parts, since the decomposition will be applied on the all parts, where for each part of the previously decomposition level will decomposed to four new parts and one of them will represent the low frequency part (ILL), i.e. for the second decomposition level there are four low frequency parts which are ( LLHHLLHLLLLHLLLL IIII 1111 ,,, ), and for the high frequency parts there will be twelve parts. Obviously, this decomposition provides possibility to utilize much more flexible fusion rules to fused result with better quality. However, more computation cost will be held in comparison with more detailed decomposition and applying more flexible fusion rules, and with the increase of decomposition level the computation cost will increase very rapidly. [4] Fusion Problem Analysis Fusing different images acquired with different sensors that work into different electromagnetic bands is very difficult task since the output of these sensors differs dramatically because each sensor is based on different physical phenomena, therefore, the relationship among their output can be complex. Such as, fusing image acquired using sensor work in the visible light band (VL) and image acquired using sensor work in the millimeter wave radar band (MMWR). [2] Due to the specific phenomenology of different sensors, each sensor will produce distinct image for the same scene. [2] for case of fusing a MMWR image with VL image, the MMWR image can contain significant features that are absent in VL image (inconsistent or complementary features), [2] beside that the MMWR image usually is high contrast image compared with the VL image, which makes the contribution of the fine details in the VL image to the MMWR image very weak using the wavelet decomposition fusion method, therefore the MMWR image features will dominate on the fused image. The Proposed Fusion Algorithm The wavelet fusion methods depend on adding the high frequency information from the high resolution image to the low resolution image, therefore it tries to decompose the 38 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 image to high frequency part and low frequency part. The high frequency part does not represent all the high frequency information that are contained in the image, but it represents a little portion of the total high frequency information in the image, and that decomposed portion becomes smaller using overlapped wavelet coefficients (like Daubechies). To perform an effective fusion method, it is needed to make the low part of the wavelet decomposition participate in the fusion rules, because even if it represents the low frequency part of the image information, but it still contains high frequency information more than the high decomposed information part. The best fusion rule to make the low decomposed frequency parts to participate in the fusion process is to use the mapping (weighted sum) fusion rule. The main obstacle to use such fusion rule is to choose of the summation weights, so that none of the fused images dominates on the resultant image, which should carry the most features of the fused images equally. Using manually selected weights will raise the weights optimality for the fusion issue, the best weights are those which consider the statistical nature of the fused images. The proposed method to calculate the weights is by using PCA analysis, where the first principle represents the statistical nature of the fused images. To perform this fusion rule on the low part of the wavelet decomposition the following steps are taken: 1- For a set of fused images Ii(x, y), i=1… D. calculate their covariance matrix Cij of DxD dimensions. 2- Calculate and sort the eigenvectors Uij according to the corresponding eigenvalues from high to low. 3- Remap the images according to the eigenvalue vector Uij. ( ) ∑ ( ) While, for the high frequency part of the wavelet decomposition, the high pass fusion rule is still valid. This paper describes a wavelet pyramid and packet based methods to decompose both low frequency part and high frequency parts at each level recursively, and then fuse the different images corresponding parts at the same level, the low frequency parts by mapping the weights (which are calculated by using PCA), and the high frequency parts by high-pass filtering (selecting the higher, replacement). After that all fused parts are restored to a fused image by inverse desecrate wavelet transform IDWT, (figure 1). illustrates the algorithm parts. Results and Discussion The above method had been validated by fusing two images for the same location but they are acquired in different wavelengths, the first acquired in the visible light band (via IKONOS) and the second acquired in the MMWR band (via IRS-1C), (fig. 2), therefore they differ in the contrast and in the spatial resolution. At first we registered these two images accurately with bias less than half pixel. The proposed method applied by decomposing the images using Daubechies Coefficients (D4), the two decomposing schemes adopted the pyramid and packet base to override the block artifact that accompanied with the use of Haar coefficients. for the pyramid decomposition schemes the proposed method tested until the seventh decomposition level while for the packet decomposition scheme we stopped until the third decomposition level. …(1) 39 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 the low frequency parts are fused using the PCA eigenvalue vector (eq. 1), while the high frequency parts fused using four fusion rule are used which are adding, replacement, selecting the higher, and pure (i.e. leaving the high frequency parts intact). The purpose of the pure fusion rule is to investigate the effect of fusion the low frequency parts only using the PCA fusion rule. What should be mentioned here that the PCA transformed had been calculated by using both the covariance matrix and the correlation matrix. To evaluate this new method, we compare its results with the results of fusing the same two images by using the high pass fusion rule only (adding, selecting, and replacement) for the pyramid decomposition scheme and for the same decomposition levels (fig. 2). The zero-mean signal to noise ratio ZMSNR are used to evaluate the resultant fused image by using this criterion as similarity index to measure the amount of information that is carried from each original image into the fused image, a good fusion result will be that which contains equal amounts of information from the original images (i.e. equal ZMSNR). Eliminating the variation in the brightness between the original images and the fused image to be effective on calculating the SNR, which is the reason for eliminating the mean brightness of the images before calculating the SNR for them. From the results (figure 3) we can see that the use of fusing the images using the high frequency parts only and for pyramid scheme only is failed and for all fusion rules because the two images (VL and MMWR) severely differ. for the propose method, we can see that the use of correlation matrix to calculate the PCA transform (figures 4 and 5) gave very good results better than the results that are obtained by using the covariance matrix to calculate the PCA transform (figures 6 and 7). This is due to the fact that the correlation matrix gives to every image a unit of variance, therefore, the differences between the original images will not affect the fusion algorithm, while the covariance matrix preserved the differences between the original images, therefore it works when the differences are small (such in fusing visible light VL image with visible near infrared VLNIF image), but for the severe cases like fusing VL with MMWR it fails. Conclusion In this paper, we have presented analyses for the image fusion techniques for the images that are taken from different sensors that depend on different physical phenomena. This analysis is based on two fusion techniques, the first is the multiresolution method (pyramid-based and packet-based wavelet transform) and the second is a statistical fusion method (PCA fusion through mapping). The results of the tests applied on the proposed fusion method that presented in this paper indicate that this method can be over the obstacle of the variation in the contrast and brightness, as it is illustrated in the first decomposition in figures (4&5), therefore the fused image represents equally the features in the SAR band image features and the visible band VL image features. The results indicate that the use of correlation matrix to calculate the PCA transform give better results for the proposed method. The other techniques that does not maintain the equality between the VL band and SAR band (figures 6&7), does not mean that it will not work with other images. References 1 – Nikolov, S.G., et al., (2004) 'Hybrid Fused Displays: Between Pixel- and Region-Based Image Fusion', In: 7 th International Conference on Information Fusion, Stockholm, Sweden, 2, 1072-1079. 2 – Pavel, M. and Sharma, R.K., (1997) 'Model-base sensor fusion for aviation', SPIE, 3088, 169-176. 40 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 3 – Zhang, Z. and Blum R.S., (1997) 'Multisensor Image Fusion Using a Region-Based Wavelet Transform Approach', DARPA IUW, 1447-1451. 4 – Jishuang, Q. and Chao, W., (2001) 'A Wavelet Package-Base Data Fusion Method for Multitemporal Remote Sensing Image Processing', SPIE, 4556, Data Mining and Applications. Registered images Wavelet Packet Decomposition Coefficients Fusion Rules Fused Images PCA Analysis Mapping Fusion Rule High-Pass Fusion Rule DWT DWT IDWT IDWT Figure (1): Flow of Wavelet Packet-based image fusion method. The registered images are decomposed by wavelet packet at first, then the low frequency parts and high frequency parts are fused by different rules, the fused image is restored by IDWT. Image-2 Image-1 Wavelet Sub-Bands Wavelet Sub-Bands 41 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 Figure( 2). Image of airport taken by two different satellites; (a) via IKONOS; (b) via IRS-1C; (c) fused image by fusing the images using the mapping fusion rules by PCA analysis only; (d) fused image by WP fusing the high frequency part only with replacement fusion rule; (e) fused image applying the proposed fusion algorithm that is mentioned in figure 1, which is obviously the resultant image which contain both images features equally better than the two previous results. (a) (b) (c) (d) (e) 42 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 -4 -2 0 2 4 6 8 10 12 14 16 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL (a) -4 -2 0 2 4 6 8 10 12 14 16 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL (b) -4 -2 0 2 4 6 8 10 12 14 16 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL (c) Figure (3) The results of applying Daubechies D4 Decomposition with different fusion rules to fuse ASAR and VL: (a) Adding (b) Replacement (c) Selection Figure (4) Results of Wavelet (Daubechies D4, pyramid)-PCA (correlation matrix) fusion method applied to fuse VL with ASAR: (a) Addition (b) Replacement (c) Selection (d) Pure 0 0.5 1 1.5 2 2.5 3 3.5 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL -2 0 2 4 6 8 10 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL 0 0.5 1 1.5 2 2.5 3 3.5 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL -2 0 2 4 6 8 10 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL (a) (b) (c) (d) 43 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 Figure (5): Results of wavelet (Daubechies D4, packet)-PCA (correlation matrix) fusion method applied to fuse VL with ASAR: (a) Addition (b) Replacement (c) Selection (d) Pure 2.65 2.7 2.75 2.8 2.85 2.9 2.95 3 3.05 3.1 3.15 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL 0 0.5 1 1.5 2 2.5 3 3.5 4 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL 2.65 2.7 2.75 2.8 2.85 2.9 2.95 3 3.05 3.1 3.15 3.2 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL (a) (b) (c) (d) 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL -4 -2 0 2 4 6 8 10 12 14 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL -2 -1 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Decomposition Level Z M S N R ( d B ) To ASAR To VL (a) (b) (c) (d) Figure (6) : Results of wavelet (Daubechies D4, pyramid)-PCA (covariance matrix) fusion method applied to fuse VL with ASAR: (a) Addition (b) Replacement (c) Selection (d) Pure 44 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 Figure (7): Results of Wavelet (Daubechies D4, packet)-PCA (covariance matrix) fusion method applied to fuse VL with ASAR: (a) Addition (b) Replacement (c) Selection (d) Pure 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 2 3 Decomposition Packet Z M S N R ( d B ) To ASAR To VLT 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL 0 1 2 3 4 5 6 7 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 2 3 Decomposition Level Z M S N R ( d B ) To ASAR To VL (a) (b) (c) (d) 45 Physics | 2016( عبو 3انعذد ) 29يجهخ إثن انهٍثى نهعهىو انصشفخ و انتطجٍقٍخ انًجهذ Ibn Al-Haitham J. for Pure & Appl. Sci. Vol.29 (3) 2016 صهر انصور انفضائية بوساطة تحويم انموجة انمعدل بوساطة تحويم انمركبات االساس حميد مجيد عبد انجبار / جبيعخ ثغذاد انهٍثى اثن –كهٍخ انتشثٍخ نهعهىو انصشفخ قسى انفٍزٌبء / 6102/اب/10,قبم في :6102/مايس/01استهم في : انخالصة اقتشاح طشٌقخ جذٌذح نصهش عذح صىس ألقًبس اصطنبعٍخ انتً تى انتقبطهب ضًن يذٌبد يختهفخ تى فً هزا انجحث تحىٌم انًىٌجخ انشقًً ألسبس انىضًن انطٍف انكهشويغنبطٍسً نتىنٍذ صىسح سيبدٌخ واحذح. تستنذ انطشٌقخ انًقتشحخ تى إعبدح احتسبة األجزاء انخبصخ ثبنتشدداد إر، تى اجشاء عًهٍخ االنصهبس ثىسبطخ قبعذتً ديج يختهفخ، انًتقطعانهشيً و انىاطئخ ثبالستعبنخ ثتحىٌم انًشكجبد األسبس انزي تى احتسبثه ثىسبطخ يصفىفخ انتغبٌش ويصفىفخ انتشاثط. وتى صهش صخ ثبنتشدداد انعبنٍخ ثىسبطخ قىاعذ صهش يختهفخ )انجًع، االنتقبء ين انتشدداد انعبنٍخ، واالثذال(، وتى األجزاء انخب انحصىل عهى انصىسح اننهبئٍخ ثتطجٍق تحىٌم انًىٌجخ انعكسٍخ. أظهشد اننتبئج انعًهٍخ صحخ انطشٌقخ انًقتشحخ نذيج هكزا شٌقخ انعبيخ نتحىٌم انًىٌجخ نذيج األجزاء انخبصخ ثبنتشدداد يع انط يتسبوٌخ ين انصىس االصهٍخ يقبسنخصىس وثنست انعبنٍخ فقط. : صهش، تحىٌم انًىٌجخ، تحىٌم انًشكجبد األسبس، هشيً، يتقطع.انكهمات انمفتاحية