Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Î Al-khwarizmi Engineering Journal Al-Khwarizmi Engineering Journal, Vol.1, No.2,pp 1-13, (2005) FFuuzzzzyy WWaavveenneett ((FFWWNN)) ccllaassssiiffiieerr ffoorr mmeeddiiccaall iimmaaggeess Dr.Walid.A.Mahmoud Electrical Engineering Department College of Engineering/ University of Baghdad!!!!!!!!! (Received 28 March 2005; accepted 11 October 2005) Abstract: - The combination of wavelet theory and neural networks has lead to the development of wavelet networks. Wavelet networks are feed-forward neural networks using wavelets as activation function. Wavelets networks have been used in classification and identification problems with some success. In this work we proposed a fuzzy wavenet network (FWN), which learns by common back-propagation algorithm to classify medical images. The library of medical image has been analyzed, first. Second, Two experimental tables’ rules provide an excellent opportunity to test the ability of fuzzy wavenet network due to the high level of information variability often experienced with this type of images. We have known that the wavelet transformation is more accurate in small dimension problem. But image processing is large dimension problem then we used neural network. Results are presented on the application on the three layer fuzzy wavenet to vision system. They demonstrate a considerable improvement in performance by proposed two table’s rule for fuzzy and deterministic dilation and translation in wavelet transformation techniques. !! Keywords: Fuzzy Theory, Neural Network ,Wavelet Transform, and Back Propagation Algorithm 1. Introduction Artificial neural networks (ANNs) have been the subjects of research for over 25 years. However, it is during the last decade or so, that research interest has blossomed into commercial application, and they are now widely used as predictive classifiers, discriminators and in pattern recognition in general. Recent neural- network research has been directed toward the improvement of the ability of multi-layer perceptron to generalize and classify data through the design of better training algorithms and superior networks. One important, yet neglected, aspect has been to understand the exact nature of the data. ANNs have been employed in the field of measurement where the nature of the data is highly diverse, ranging from digital pixel values from CCDs in vision systems to analogue D.C. Conductance signals in a semi-conducting oxide electronic nose.. The uncertainty in the data comes in as apart of the real-world implementation itself, often attributed solely to the imprecision of the measurement. Conventional ANNs (e.g. multi-layer perceptron) do not attempt to model the data to the another domain wave domain which constraints by fuzzy rules.. This often culminates in poorly trained networks where the problem because more significant as the uncertainty in the data increase and the size of the training set decreases. Fuzzy neural networks (FWNs) make use of base mother matrix (such as Dr.A.barsoum Electrical Engineering Department University of technology Entather Mahos Electrical Engineering Department University of technology This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ï Morelet) and fuzzy logic to model data. FWNs have a relatively recent history but interest has increased through the application of wavenet and fuzzy logic in computer vision system. [1,2]. 2. Artificial neural networks ANNs are mathematical constructs that try to mimic biological neural systems. Over the years, ANNs have become recognized as powerful pattern recognition techniques. The networks are capable of recognition spatial, temporal or other relationships and performing tasks like classification, prediction and function estimation. ANN development differs from classical programming in the fact that in modality the data variance is learnt over a number of iterations. One of the main problems of an ANN approach is knowing when optimal network parameters have been found. Further, as the data sets become less well behaved, the training typically becomes more difficult, and the class prediction less than satisfactory. It is generally accepted [8] that there are several advantages in applying ANNs as opposed to any other mathematical or statistical techniques. For instance, their generalization abilities are particularly useful since real-world data are often noisy, distorted and incomplete. In addition, it is difficult to handle non-linear interactions mathematically. In many applications, the systems cannot be modeled by other approximate methods such as expert systems. In cases where the decision making is sensitive to small to small changes in the input, neural networks play an important role. Nevertheless, ANNs have some potential disadvantages as well, since the choice of the way in which the inputs are processed is often largely subjective and different results may be obtained for the same problem. Furthermore, deciding on the optimal architecture and training procedure is often difficult, as stated above. Many problems would need different subjective considerations, including speed, generalization and error minimization. ANNs have other potential disadvantage as well. For example, there is little formal mathematical representation of their decision and this been a major hurdle in their application in high-integrity and safety critical systems. Multi-layer perceptron are most commonly used ANN in pattern classification and typically comprise as input layer, an output layer and one or more hidden layers of nodes. Most of our vision system has employed three layer networks (input, one hidden, and output layer), since the addition of further hidden processing layers does not provide substantial increases in discrimination power [9]. We have used an BP algorithm to learning neural networks which is used in our computer vision systems. 3. Wavelet Transform: - In this section, some basic concept in wave let transform are briefly recalled. A. The continues Wavelet Transform:- The continuous wavelet transform is the first studies transform. We summarized some result on continuous wave let in L2(Rd), where index d stands for dimensions. If a function dRx )( is radial, its Fourier transform )(^  is radial as well. The transform )(x is admissible as a mother wavelet function if 1 0 2 )(^)2(     dC if function  satisfy condition(1), for any function 22 )(RLf  , its continuous wavelet transform defined as[9].   dR dx a bx xfdaba f w )().(2/),(  B. The Discrete Wave let Transform:- The continues wave let transform and its inverse transform are not directly on digital computer. It must be discredited. This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ð When wavelet transform is discredited, some condition should be satisfied so that this discrete version of the reconstruction of f can be actually hold. The corresponding family of dilated and translated wave lets [10]. (3) ,:2/ dznZm ma nbxd ma                    4. Fuzzy Theorem: Fuzzy logic is a powerful technique for problem solving which has found widespread applicability in the areas of control and decision making. Fuzzy logic was invented by Zadeh in 1965 and has been applied over recent years to problem that are difficult to define by precise mathematical models. The approach is particularly attractive in the field of decision making, where information often has an element of uncertainty in it.[4] The theory of fuzzy logic in run relates to the theory of fuzzy sets where an effort is made to distinguish between the theory of probability and possibility. There is more than one way in which fuzziness can be introduced into neural networks and hence different works mean different things by the term ‘fuzzy neural network”. Since researches define these networks as having fuzzy inputs and fuzzy outputs and hence try to fuzzify (i.e., assign a membership value to data values within the range 0-1 using a possibility distribution) before data are presented to the ANN. This concept can obviously be further extended, as described, for example, by Zadeh [5], where the inputs and outputs are truly fuzzified by their transformation into linguistic terms. So rather than having a particular numerical value (e.g., in the input or output), we can describe values linguistically as very low, low, moderate, high, very high, etc. This kind of fuzzification, though tempting for some application (e.g., classifying the x- ray images), would not be suitable for others in which the boundaries are hard to specify. Fuzzy logic attempts to distinguish between possibility and probability as two distinct theories governed by their own rules. Probability theory and Bayesian networks can be used where the events are receptive and statistically distributed. The theory of possibility is more like a membership-class restriction imposed on a variable defining the set of values it can take. In the theory of probability, for any set A and its complement Ac, A Ac= null set), which is not true in the case of the theory of possibility. Possibility distributions are often triangular and so similar in shape to normal distribution with mean value having the highest possibility of occurrence, which is one. Any value outside the min-max range has a possibility of occurrence of zero. Hence in mathematical terms, the possibility that aj is a member of the fuzzy set X={ a1 ,a2 …… an} is denoted by its membership value M (aj ). This membership value of aj in X depends upon the mean, minimum and maximum of the set X. An introductory treatment to the theory of fuzzy logic is given by McNeill and Freiburger [6]. A more mathematical description of fuzzy sets and the theory of possibility are available in Dubois and Prade [7]. This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ñ 5. Fuzzy Wavenets Recently, simple and efficient wavelet-based methods have been developed for online learning. These fuzzy wavenets methods combine wavelet theory to fuzzy logic and neural networks. They permit to determine and validate adaptively appropriate fuzzy rules in online problems. The model is refined as more data are furnished to the system. With only a few data points, the information on the underlying surface is small and a low- resolution description of the system is appropriate, while with an increasing number of data points, a higher resolution may be justified. New rules are added to the description of the surface with increasing data points. Using an automatic procedure based on the fast wavelet decomposition and reconstruction algorithm validates the rules. Learning is fully automatic and does not require any external intervention, making these methods very useful in practical applications, [3]. Function, S Measurement, Y (a) Figure. 1. (a) The possibility distribution S (v:X,B,Y).(b) The membership function M is related to the function S . X B!! Y!! 0!! 0.5!! 1!! B Y Y+B/2Y-B/2 Measurement, Y 1 0 (b) This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ò In figure (2) the first layer of FWN is the input Layers, which introduces inputs into network, the second Layer of FWN is the membership layer, wavelets is used as membership function. We use Mexican hat function as mother wavelet, function)hat(MexicanPolywog4 )2x5.0(exp)2x1()x(  Then the membership function of each linguistic variable can be achieved by translating and dilating the mother wavelet. The third Layer of FWN is the rule layer. Figure. 2. Structure of FWN (fuzzy wavenet). X1!! Xn!! P Z!! N!! Input layer!! Rules layer: P Positive, Z Zero, N Negative Y1!! Yj!! Ym!! Zk!! Z1!! Member ship function layer!! Output layer This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ó 6. Data base Library:- Figure 3. Samples from data base medical images. (a) First image Germinomal!! (b) Second image mixe germ cell tumors (c) Third image Mixed (d) Fourth image Plasmacytoma!! (e) Fifth image Leukemia!! (f) Sixth image Malignant!! (j) Seventh image Chwannoma!! (k) Eighth image Hemangioperictoma!! (l) Ninth image Dermoid cyst!! (z) Tenth image Neuoglial cyst!! This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ô The algorithm of Fuzzy Wavenet (FWN) classifier for medical images applied as: Start!! Enter Learning rate momentum rate initial seed integer for MLP!! Random function to return the random value between 0 to seed integer!! While TSSE >0.01 !! Enter file images (size 100x100) gray scale or color the calculate member ship function Calculate fuzzy rule layer Calculate actual output For every input enter target TSSE=0.5(Tj-Yk)^2!! Save the total weight!! End!! Update the weights P!! Z!! N!! t0, d0 t1, d1 t2, d2 Figure 4. The step of the algorithm Experiment results data base. This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Õ Where: P: represent positive fuzzy rule; N: represent Negative fuzzy rule; Z: represent zero fuzzy rule. t0,d0 ……. T3,d3 represent translation and dilatation in each fuzzy action. TSSE: total sum square error. 7. Experiment results: - We propose two types of rules FWN, which contain the member ship function, depend on the three rules. Number of hidden input nodes = 5050(2500), number of hidden nodes 7500, output nodes=10.the rules determine by the following table with member ship function in Fig. 5: - Previous\actual N Z P N P Z N Z Z Z Z P N Z P !! Table 1: Sets of fuzzy Rule. Figure 5: Member ship function Considered data base library with tenth images has (5050) pixels, 2500 units for input layer, and 10 for out put layers, with tested number of hidden layer. Initialization weight from random function given value between (-1 to 1). Training parameters for first data base library Ten gray scale images (5050) hidden = (input + output)/2 Desired accuracy=90% three fuzzy rules Morelet bases function (x)=(1-x2)exp(-0.5*x2), Wavelone=100,x(new)=(xold-t/d). X!! (X) Z!!N!! P This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Ö TSSE 0.09 0.097 0.096 0.093 0.08 0.06 No. iteration 208 29 22 7 4 4 Learning rate 0.01 0.075 0.1 0.35 0.75 0.9 For optimal value for learning rate=0.9, with minimum error and hidden nodes =7500 units show figure below: - TSSE 1.0 0.584 0.267 0.118 0.057 No. iteration 0 1 2 3 4 weight 0.0302 0.0512 0.0639 0.0705 0.0775 !! Table 2: Learning Rate 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.5 1 1.5 2 2.5 3 3.5 4 No. iteration T S S E + w e ig h t TSSE weight Figure6: Relation ship between TSSE and number of iteration for three rules FWN!! Table 3: initial weight This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Î Í To get more size such as images with 100100, need to design the greater Than three rules. We proposed FWN with seven rules shown in table 3: - Actual/previous NB N SN Z SP P PB NB PB P SP Z N N NB N P P SP Z SN N NB SN SP SP SP Z SN N NB Z Z Z Z Z Z Z Z SP N SN SN Z SP P P P N N N Z P PB PB PB NB NB N Z P PB PB Table4: - Seven rules for FWN. Considered data base library with tenth images has (100100) pixels, 10000 units for input layer, and 10 for out put layers, with tested number of hidden layer. (X) X!! Z!!SN!!N!!NB!! SP!! P!! PB!! Figure 7: Member ship function for seven rules This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Î Î Initialization weight from random function given value between (-1 to 1). Training parameters for first data base library Ten gray scale images (100100) Desired accuracy=90% Seven fuzzy rules poly 4 bases function (x)=(1-x2)exp(-0.5*x2), Wavelone=100,x(new)=(xold-t/d). TSSE 0.099 0.099 0.098 0.0883 0.095 0.069 No. iteration 359 49 37 12 6 6 Learning rate 0.01 0.075 0.1 0.35 0.75 0.9 For optimal value for learning rate=0.9, With minimum error and hidden nodes =10000 units show figure below:- TSSE 1.0 0.617 0.403 0.252 0.161 0.104 0.069 No. iteration 0 1 2 3 4 5 6 weights 0.056 0.14 0.225 0.304 0.307 0.434 0.484 !! !! !! Table 4: Learning Rate!! 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 1 2 3 4 5 6 No.ite ra tion T S S E + w e ig h t TS S E weight Figure.8: Relation ship between TSSE and number of iteration for seven rule FWN!! This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Î Ï 8. Discussions: Wavelet network are an alternative to sigmoid neural networks, but if add the fuzzy theory to design. The expert system which classifier the medical images with desired accuracy equal 90%, with noisy image equal 30%. For large size imaging we need to design more than fuzzy rule to deterministic the values of translation and dilation. Choice for mother matrix effect on the results as number of iteration lees or greeted. The powerful for such system vision can be analysis huge data and give classified results for any small region area inside the pictures. This is most problem severed the doctor in classified or recognition the decease. The FWN don’t effect by varying number of hidden nodes because this values is constant depend on number of rules. Also the learning time in FWN is very shorter if comparing with another network. The learning time of FWN decrease by 50 % with desired accuracy over than 90%. More problem was solved by FWN such as local minimum. References: - [1] J-N. Hwang, S-R. Lay,”Regression Modelling in Back-Propagation Pursuit Learning”,IEEE Transaction on Neural Networks.pp.342-353, 1994. [2] A.Judisky, Q. Zhang,”Wavelets in Identification: wavelets, splines, neurons, fuzzes: how good for identification?”, Rapport INRIA No. 2315, 1994. [3] Y. Oussar, I. Rivals,” Training Wavelet Network for Non linear Dynamic Input- Output Modeling “, Neurocomputing (1-3), pp. 173-188, 1998. [4] Chang, P.R., Yeh, B.F.,“Nonlinear communication channel equalization using wavelet neural network“, Proc.IEEE Int. Conf. on Neural Networks: IEEE World Congress on Computational Intelligence, IEEE, pp.3605-3610 (1994). [5] Chang, C.S., Weihui Fu, Minjun Yi,“Short term load forecasting using wavelet networks“, Engineering Intelligent Systems for Electrical Engineering and Communications 6, 217-23 (1998). [6] Daugmann,J., “Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression“, IEEE Trans. Acoust., Speech, Signal Proc. 36, 1169-1179 (1988). [7] Dickhaus, H., Heinrich, H.,“Identification of high risks patients in cardiology by wavelet networks“, Proc. 18 Th IEEE Engineering in Medicine and Biology, 31 Oct-3 Nov. 1997, IEEE, vol. 3, pp.923-4 (1996). [8] Echauz, J., "Strategies for Fast Training of Wavelet Neural Networks," 2nd International Symposium on Soft Computing for Industry, 3rd World Automation Congress (WAC'98), Anchorage, Alaska, May 10-14, 1-6 (1998). [9] Thuillard, M.,“Applications of wavelets and wavenets in soft computing illustrated with the example of firedetectors“, SPIE Wavelet Applications VII, April 24-28 2000 (Orlando), in press (2000b). [10] Yeung, L.F., Li, X.W.,“Multi-input system identification and its applications using wavelet constructive method, Proc. 35 th IEEE Conf. On Decision and Control, 11- 13 Dec. 1996, Kobe, IEEE, vol. 3, pp.3230- 5 (1996). This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/ Dr.Walid.A.Mahmoud /Al-khwarizmi Engineering Journal, vol.1, No. 2, PP 1-13 (2005) Î Ð !! !! ! !! ! ƒ! ! !Œ!¾ƒ! š!!Œ!! ƒ!!!!!Œ!! !! œ!!! ƒ!! !Œ!! ! ! !Œ!! ƒ!! !! !! ! !Œ!! !!!! ! ! !! !!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ! !!!! ! ! !!!!!! ! !!! ! !!!!!! !!!!! !!!!!!!!!!!!!!!!!!!! !!!!! ! !!!!! ! !!!! ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!! ! !!!!! ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!! ! !!!!! ! !!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ! !! !! !!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!! !!! !! !!!!!!!!! !!! ! ! ! ! !Œ!!!!!!! !!!! !!! Ÿ!!!! !! !! !!! !! Š!! ! !!!! !!! ! !! !!!! ! Š!!! !!!!!!! »!Š!! ! !!!! !! !»! !!! !! ! !!! !!!! »ž!! »!!Ÿž! ! !!!! ! ! Š!! ! !!!! ! !!Ÿ! !!!! !! !!! !! !!!! ! !!! ! !! !! !! !! ! !! !Š!! ! !!! ! !! !! !!! ! !! ! !! ! !!! ! !!!! !!!! !!!š!Š!!!Ž !! Ÿ!!!! !Ž !!! !!!!¿! !! ! !¿! !!! ! !! !! !!! !Š!! ! !!!! ! !!!!! !!!¿! Ÿ!!!!!! !! ž!!! ! ! !! !!!! !Š!! ! !!!! ! !! !!!! ! !!š!! !!!!FWN!!!!!!!!! !»! !!!!! !! !»!!! ! ! !!! !!!! ! ! !!! !!!! ! ! !!!Ž !!! !!! ! ! ! Ÿ!!! ! Š!! !!!! ! !! ! ! !! !!!!!!! ! »! !!! »!!! ! ! ! !!! !! !¿!!! !! !!!! !!!!!!! ! ž ! !!! !!!!!! Š!!!!! ! ! !! !!!! !Š!! ! !!!! ! !! !! !!!!!!! !!!! ! !! ! Š!!! !! !!! ! !¿! !!Š!Ž !!! !!!!!!!!!!!!!!! !! !! !!!! !!Ÿ!!! ! ! !!!!!! !¿! !!!! !! ! !Ÿ! !!!!!!! »!!!!!!!»Š!! ! !! !Š!! ! !!!!!!! !! !!!¿!! ! !!!!! !!!Ž ! Ÿ!!! ! ! !Ž ! ! !! !!!!! !!! !¿!Š! !! !!!! !! ! ! !! ! !! !! ! ! !!!! Š!!Ÿ! !!!! ! !! !! !! ! !!!¿!Š! !!!! !!!¿! !! ! !!! !! !!!! !! !!! Ÿ!!!! ! !! !!!!!! !! !!!!!! ! !!»Ÿ! !!!!!! ! !! !!! !!! ! !! !!!!! !! !!¿! ! !! ! !! œ!!!!! !! !¿! ! ! !!!!!! ! ! !!! ! ! !! !!!! !Š!!!!!!!! »! !! »!!!!¿! !!»Š!!!! ! !! !! ! !!! !! !!! !!!!! ž!!! !Ÿ!!! œ!!!!! !! !¿! ! ! !!!!!! !! !Š!! ! !!!! !!!!!!!!!!! !! !!!! !! !!! !! !! !!!!!!! ! !!!!!!! !!!!!!!! !! ! !! ! !! !!!!!! !! !!!!!!! This page was created using Nitro PDF trial software. To purchase, go to http://www.nitropdf.com/ http://www.nitropdf.com/