Microsoft Word - MATH041130-corrected.doc 69-77 SQU Journal For Science, 11 (2006) © 2006 Sultan Qaboos University 69 Two New Measures of Fuzzy Divergence and Their Properties Om Parkash*, P.K. Sharma** and Subhash Kumar*** Department of Mathematics,Guru Nanak Dev University,Amritsar- 143005,Email:omparkash777@yahoo.com*,Hindu College, Amritsar- 143001,Email: pk_sharma7@rediffmail.com**, S.S.M. College, Dinanagar***. قياسان جديدان للتباعد المشوش وخواصهما شارما و سبهاش كومار. ك.باركاش ، ب. أوم في هذا البحث يتم . في األبحاث ايجاد كثيراً من قياسات التباعد الموجه وقياسات التباعد المشوش المقابل لهملقد تم :خالصة . المطلوبةتطوير قياسين جديدين للتباعد المشوش ، كما يتم مناقشة خواصهما ABSTRACT: Several measures of directed divergence and their corresponding measures of fuzzy divergence are available in the exiting literature. Two new measures of fuzzy divergence have been developed and their desirable properties have been discussed. KEYWORDS: Fuzzy set, Fuzzy divergence, symmetric divergence, convex function. 1. Introduction he concept of distance has been proved to be very important for its applications to science and engineering. Naturally, attempts were made to extend the concept of distance for applications to problems in economics, sociology, psychology, linguistics, genetics, biology, etc. It was however soon realized that the concept of distance would need a modification for such applications. To explain the necessity for a new concept of distance for social, economics, statistical, business and management sciences, we consider the following typical problems in these sciences: 1. Find the distance between the intellectual attainment of two students whose proportions of grades A,B,C,D,E are ( )1 2 3 4 5, , , ,p p p p p and ( )1 2 3 4 5, , , ,p p p p p , respectively. 2. Find a measure for the variation in the intellectual attainments of a class of m students when the proportions of grades of the ith student are given by ( ) ( )1 2, ,...., ; 1, 2,...,i i nip p p i m= . 3. Find the distance between income distributions in two countries whose proportions of persons in n groups are ( )1 2, ,...., np p p and ( )1 2, ,...., nq q q . T OM PARKASH, P.K. SHARMA AND SUBHASH KUMAR 70 4. Find the distance between balance sheets of two companies or between the balance sheets of the same company in two different years. . 5. Find a measure for the improvement in income distribution when income distribution ( )1 2, ,...., nq q q is changed to ( )1 2, ,...., nr r r by some government measures when the ideal distribution is assumed to be ( )1 2, ,...., np p p . 6. Find a measure for comparing the distribution of industry or poverty in different regions of a country. 7. Find the measure of genetic distance between two breeds of animals or between two species of plants. A measure D(P: Q) of divergence or distance or cross entropy or directed divergence which has found deep applications in many disciplines can be defined as the discrepancy of the probabilities distribution P from another probabilities distribution Q. In some sense, it measures the distance of P from Q. The most useful measure of directed divergence is due to Kullback and Leibler (1951). It is given by ( ) 1 : In n i i i i p D P Q p q= = ∑ (1.1) Distance measure is a term that describes the difference between fuzzy sets. Liu (1992) gave the axiom definition of distance measure and discussed the relationships between distance measure and fuzzy entropy. Distance measure can be considered as a dual concept of similarity measure. Many researchers, such as Yager (1979), Kosko (1992) and Kaufmann (1975) have used distance measures to define fuzzy entropy. Liu (1992) extended Yager’s formula to give a general relationship between distance (or similarity measure) and fuzzy entropy and obtained some important conclusions. Zadeh (1968) introduced the concept of fuzzy sets in which imprecise knowledge can be used to define an event. Using the concept of fuzzy message conditioning, a fuzzy distance measure between two fuzzy sets has been suggested by Bhandari and Pal (1993). Bhandari and Pal (1993) introduced a measure corresponding to Kullback and Leibler’s (1951) probabilistic directed divergence given by ( ) ( ) ( ) ( ) ( )( ) ( ) ( )1 1 I A:B log 1 log 1 n A i A i A i A i i B i B i x x x x x x µ µ µ µ µ µ= ⎡ ⎤− = + −⎢ ⎥ −⎣ ⎦ ∑ (1.2) where ( )A ixµ gives the degree of belongingness of the element ix to the set A. The symmetric fuzzy divergence between two fuzzy sets A and B is given by ( ) ( ) ( )J A:B =I A:B +I B:A (1.3) Since there are non-exponential models, innovation diffusion models, epidemic models, a variety of models in Economics, Social Sciences, Biology or even in Physical Sciences, we need a variety of information measures for each field to extend the scope of their applications. Hence the development of new generalized parametric and non-parametric measures of divergence is important for the fuzzy distributions. In this paper, we have developed such important measures of fuzzy directed divergence. In section 2, we propose two new measures of fuzzy directed divergence and prove their validity. Some desirable properties of these measures have also been studied. In section 3, the proposed measures are generalized. 2. Two new measures of fuzzy divergence We now propose a new measure of fuzzy divergence which corresponds to Ferreri’s (1980) measure of directed divergence for a probability distribution. This measure is given by TWO NEW MEASURES OF FUZZY DIVERGENCE AND THEIR PROPERTIES 71 ( ) ( )( ) ( ) ( ) ( )( ){ } ( )( )( )( ) 1 1 1 log 11 : ; 0 1 1 1 1 log 1 1 A i A i n B i a i A i A i B i a x a x a x I A B a a a x a x a x µ µ µ µ µ µ = +⎡ ⎤ +⎢ ⎥+⎢ ⎥= >⎢ ⎥+ − ⎢ ⎥+ + − + −⎢ ⎥⎣ ⎦ ∑ (2.1) We know that ( ) ( )( ) ( ) ( )( ) ( ) 0 1 0 0 1 /1 log 0 1 / n A iA i i B i a x n aa x n a a x n a µ αµ α µ β= ⎡ ⎤+ ++ ≥⎢ ⎥ + + +⎢ ⎥⎣ ⎦ ∑ where ( ) ( )0 0 0 0 1 1 ; ; , n n A i B i i i x xµ α µ β α β = = = =∑ ∑ may be different from unity, or ( )( ) ( ) ( ) ( ) 00 1 0 1 1 log log 1 n A i A i i B i a x n a a x n a a x n a µ α µ α µ β= + + + ≥ + + + ∑ (2.2) Similarly, ( )( ){ } ( )( )( )( ) ( )( ) ( ) ( ) 1 0 0 0 1 1 1 1 log 1 1 1 1 log 1 n A i A i i B i a x a x a x n a a n a a n a a µ µ µ α α β = + − + − + − + − ≥ + − + − ∑ (2.3) Adding (2.2) and (2.3) and dividing by a , we get ( ) ( )0 0: ,aI A B f α β≥ where ( ) ( ) ( )( ) ( ) ( ) 00 0 0 0 0 0 0 11 , log 1 log 1 n a an a f n a n a a a n a n a a αα α β α α β β ⎧ ⎫+ −+⎪ ⎪ = + + + −⎨ ⎬ + + −⎪ ⎪⎩ ⎭ (2.4) Differentiating (2.4) w.r.t. 0a , we have ( ) ( ) ( ) 00 0 0 0 0 1 , log log 1 n a an a f n a n a a αα α β β β + −+ ′ = − + + − Also ( ) ( ) ( ) ( )( )0 0 0 0 2 , 0 1 an a f n a n a a α β α α + ′′ = > + + − OM PARKASH, P.K. SHARMA AND SUBHASH KUMAR 72 Thus ( )0 0,f α β is a convex function of 0α which has its minimum value when 0 0 0 0 2 1 2 n a n na a n na n a n na a n na α α β β + + − + = = = + + − + that is , when 0 0n a n aα β+ = + or when 0 0α β= and the minimum value is 0 so that ( )0 0, 0f α β > and vanishes when 0 0α β= . Consequently, ( ):aI A B is a convex function of ( ) iA ixµ ∀ . Again, differentiating (2.4) w.r.t. 0β , we have ( ) ( ) ( ) 00 0 0 0 0 1 , 1 n a an a f n a n a a αα α β β β + −+ ′ = + + + − Also, ( ) ( ) ( ) ( )( ) ( )( ) 00 0 0 2 2 0 0 1 , 0 1 a n a aa n a f n a n a a αα α β β β + −+ ′′ = + > + + − Thus ( )0 0,f α β is a convex function of 0β and consequently, ( ):aI A B is a convex function. Thus for all values of 0α and 0β , we have (i) ( )A:B 0aI ≥ (ii) ( )A:B 0 when A=BaI = (iii) ( ):aI A B is a convex. Hence ( )A:BaI is a valid measure of fuzzy directed divergence and ( ) ( ) ( )A:B A:B B:Aa a aJ I I= + (2.5) is a valid measure of fuzzy symmetric divergence . Now ( ) ( ) ( ) 0 A:B 0 and A:B I A:Ba aa a Lt I Lt I → →∞ = = Thus ( ) ( ) ( ) min max : 0 and : I A:BI A B I A B⎡ ⎤ ⎡ ⎤= =⎣ ⎦ ⎣ ⎦ Monotonicity of ( ):aI A B Differentiating (2.1) both sides w.r.t. ,a we get TWO NEW MEASURES OF FUZZY DIVERGENCE AND THEIR PROPERTIES 73 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 2 1 2 1 1 11 : log 1 1 1 1 11 log 1 1 1 n A i A i i B i B i n A i A i i i B i a x a xd I A B da a a x a x a a x a a x a a a x a a xβ µ µ µ µ µ µ µ µ = = ⎡ ⎤+ + = − − +⎢ ⎥ + +⎣ ⎦ ⎡ ⎤+ − + + − − +⎢ ⎥ + − + +⎢ ⎥⎣ ⎦ ∑ ∑ (2.6) We know that log 1 0x x− + ≤ . Thus ( ) ( )ψ log 1 0x x x= − − + ≥ So that ( ) 1 1ψ 1 1x x x ⎛ ⎞′ = − − = −⎜ ⎟ ⎝ ⎠ . If ( )0 1, ψ 0x x′< < < and if ( )1, ψ 0x x′> > and if ( )1, ψ 1 0x = = so that the graph of ( )ψ x is as shown in Figure 1. Thus ( ): 0a d I A B da ≥ so that ( )A:BaI is an increasing function of a . Hence as a increases from 0 to ∞ , ( )A:BaI increases from 0 to ( )I A:B . OM PARKASH, P.K. SHARMA AND SUBHASH KUMAR 74 Now we propose another measure of fuzzy directed divergence which include fuzzy directed divergence corresponding to Kullback and Leibler’s (1951) probabilistic measure and fuzzy directed divergence corresponding to Kapur’s (1985) measure of probabilistic directed divergence in particular or in limiting cases. This measure is given by ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( )( ) ( ) ( ) ( )( ) ( )( ) ( )( ) 1 1 1 : log 1 log 1 1 111 1 log 1 1 log 1 1 1 n A i A i a A i A i i B i B i n A iA i A i A i i B i B i x x I A B x x x x a xa x a x a x a a x a x µ µ µ µ µ µ µµ µ µ µ µ = = ⎡ ⎤− = + −⎢ ⎥ −⎣ ⎦ ⎡ ⎤⎧ + −+ ⎪ − + + + −⎢ ⎥⎨ + + −⎢ ⎥⎪⎩⎣ ⎦ ∑ ∑ (2.7) We know that ( ) ( ) ( ) 0 1 0 0 / log 0 / n A i A i i B i x x x µ µ α α µ β= ≥∑ or ( ) ( ) ( ) 0 0 1 0 log log β n A i A i i B i x x x µ α µ α µ= ≥∑ (2.8) Similarly, ( )( ) ( ) ( ) ( ) 00 1 0 1 1 log log 1 n A i A i i B i x n x n x n µ α µ α µ β= − − − ≥ − − −∑ (2.9) Adding (2.8) and (2.9), we get ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1 0 0 0 0 0 0 1 log 1 log 1 log log n A i A i A i A i i B i B i x x x x x x n n n µ µ µ µ µ µ α α α α β β = ⎡ ⎤⎛ − + −⎢ ⎥⎜ −⎢ ⎥⎝⎣ ⎦ − ≥ + − − ∑ (2.10) Adding (2.4) from (2.10), we get ( ) ( )0 0: ,aI A B g α β≥ where ( ) ( ) ( ) ( )( ) ( ) ( ) 0 0 0 0 0 0 0 0 00 0 0 0 0 , log log 11 log 1 log 1 n g n n n a an a n a n a a a n a n a a α α α β α α β β αα α α β β − = + − − ⎫⎧ + −+ ⎪ + + + + −⎨ ⎬ + + − ⎪⎩ ⎭ (2.11) TWO NEW MEASURES OF FUZZY DIVERGENCE AND THEIR PROPERTIES 75 Differentiating (2.11) w.r.t. 0α , we have ( ) ( ) ( ) 00 0 0 0 0 0 0 0 0 1 , log log log log 1 n a an n a g n n a n a a αα α α α β β β β β + −− + ′ = − + − − + + − and ( ) ( ) ( )( )( )0 0 0 0 0 0 2 , 0 na a g n n a n na a α β α α α α + ′′ = > − + + − so that ( )0 0,g α β is a convex function of 0α which has its minimum value when 0 0α β= , that is when ( ) ( )A i B ix xµ µ= , and the minimum value is 0 so that ( )0 0, 0g α β ≥ and vanishes when 0 0α β= . Consequently, ( )A:BaI is a convex function of ( )A ixµ and similarly we can prove that ( ):aI A B is a convex function of ( )B ixµ . Thus for all real values of 0α and 0β , we have (i) ( )A:B 0aI ≥ (ii) ( )A:B 0aI = when A = B (iii) ( )A:BaI is convex so that ( )A:BaI is a valid measure of fuzzy directed divergence and ( ) ( ) ( )A:B A:B B:Aa a aJ I I= + is a valid measure of symmetric divergence. The following relations are obvious : ( ) ( ) ( ) ( ) ( ) A:B (A:B) A:B A:B A:B A:B a a a a I I I J J J = − = + Also ( ) ( ) 0 A:B I A:Baa Lt I → = 3. Generalized measure of fuzzy divergence A generalization of Kullback Leibler’s (1951) measure is given by ( ) ( ) ( ) ( ) ( )( ) ( ) ( )1 1 : 1 1 n i i B i B i i B i B i x x I A B x x x x µ µ µ µ µ µ Α Α = ⎡ ⎤⎛ ⎞ ⎛ ⎞− ′ = φ + − φ⎢ ⎥⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟−⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦ ∑ (3.1) where ( )φ ⋅ is a twice differentiable convex function with ( )1 0φ = . The measure (1.1) can be obtained by taking ( ) logx x xφ = in (3.1) OM PARKASH, P.K. SHARMA AND SUBHASH KUMAR 76 Similar generalizations of ( )A:BaI and ( )A:BaI are given by ( ) ( )( ) ( ) ( ) ( )( ){ } ( )( )( )( ) 1 1 1 : ; 0 1 1 1 A i B i n B i a i A i B i B x a x x I A B a a x a x x αµ µ αµ α αµ µ α µ = ⎡ ⎤⎛ ⎞1+ + φ⎢ ⎥⎜ ⎟ 1+⎢ ⎥⎝ ⎠ ′ = >⎢ ⎥⎛ ⎞1+ 1−⎢ ⎥+ + − φ⎜ ⎟⎢ ⎥⎜ ⎟1+ −⎝ ⎠⎣ ⎦ ∑ (3.2) and ( ) ( ) ( ) ( ) ( )( ) ( )( ) ( )( )1 : 1 1 n ii a B i B i i i B i xx I A B x x x x ΑΑ Β µµ µ µ µ µ= ⎡ ⎤⎛ ⎞1−⎛ ⎞ ′ = φ + − φ⎢ ⎥⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟−⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦ ∑ ( )( ) ( ) ( ) ( )( ){ } ( )( ) ( )( ) 1 1 1 1 1 ; 0 1 1 A i B i B i n B i i A i B i x a x a x x a a x x αµ µ µ αµ α µ α µ = ⎡ ⎤⎛ ⎞1+ + φ + + −⎢ ⎥⎜ ⎟⎜ ⎟1+⎢ ⎥⎝ ⎠ − >⎢ ⎥ ⎡ ⎤1+ −⎢ ⎥+φ⎢ ⎥⎢ ⎥1+ −⎢ ⎥⎣ ⎦⎣ ⎦ ∑ (3.3) Now differentiating (3.2) w.r.t. a , we get, ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( )( ) ( )( ) ( )( ) ( )( ) ( )( ) 2 1 1 1 1 1 1 1 1 1 1: 1 1 1 1 1 1 1 1 1 1 1 1 A i A i A i B i B i B i n A ia i B i A i A i B i B i a x a x a x a x a x a x a xd I A B da a a x a x a x a x a x µ µ µ µ µ µ µ µ µ µ µ µ = ⎡ ⎤⎛ ⎞ ⎛ ⎞ ⎛ ⎞+ + + ′⎢ ⎥−φ + − φ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ + + +⎢ ⎥⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎢ ⎥ ⎛ ⎞+ −′ ⎢ ⎥ = +φ⎜ ⎟⎢ ⎥⎜ ⎟+ −⎝ ⎠⎢ ⎥ ⎢ ⎥⎛ ⎞ ⎛ ⎞+ − + −⎢ ⎥′+ − φ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟⎢ ⎥+ − + −⎝ ⎠ ⎝ ⎠⎣ ⎦ ∑ (3.4) If we take ( ) ( ) ( ) ( ) ( ) ( )1 1 ,h x x x x x x′ ′= −φ + − φ + − φ then ( ) ( ) ( )1h x x x′ ′′= − φ Since ( ) 0x′′φ > ( ) if 0 1h x x′ < < < and ( ) 0 if 1h x x′ > > TWO NEW MEASURES OF FUZZY DIVERGENCE AND THEIR PROPERTIES 77 Also since ( ) 0,φ 1 = we have ( )1 0h = . Thus equation (3.4) gives that ( )( ): 0a d I A B da ′ > Thus ( )A:BI′ is an increasing function of a . Also, ( ) 0 : 0 a Lt I a A B → ′ = and ( ) ( ): :aaLt I A B I A B→∞ ′ ′= 4. Conclusion It has been observed that there are redundancies and overlapping in similar situations which if removed can increase the efficiency of the process. The development of new parametric and non-parametric measures of information will definitely reduce uncertainty and consequently the process can be more efficient. Thus development of new generalized probabilistic and fuzzy measures of divergence is necessary to get as much insight as possible into the various physical situations. Many such fuzzy divergence measures can be generated for different situations. 5. Acknowledgement One of the authors, P.K. Sharma expresses his warm thanks to the University Grants Commission for providing financial assistance for the preparation of this paper. 6. References BHANDARI, D. and PAL, N.R. 1993. Some new information measures for fuzzy sets. Infor. Sci., 67: 209-228. FERRERI, C. 1980. Hyperentropy and Related Heterogeneity, Divergence and Information Measures. Statistica, 40(2): 155-168. KAPUR, J.N. 1985. Some new measures of entropy and divergence. Jour Math. Phy. Sci., 19(6): 489-501. KAUFMANN, A. 1975. Introduction to the Theory of Fuzzy Subsets. Academic Press, New York. KOSKO, B. 1992. Neural Networks and Fuzzy Systems. Prentice Hall, Englewood Cliffs, N J. Received 30 November 2004 Accepted 22 August 2006