D:\sbornik\...\Article.DVI


Mathematical Problems of Computer Science 40, 34{38, 2013.

On N eyman-P ear son P r inciple in M ultiple H ypotheses

T esting

E vg u e n i A . H a r o u t u n ia n a n d P a r a n d z e m M. H a ko b ya n

Institute for Informatics and Automation Problems of NAS RA

e-mail: eghishe@ipia.sci.am, par h@ipia.sci.am

Abstract

The aim of this paper is to newly generalize the classical Neyman-Pearson Lemma
to the case of more than two simple hypotheses.

Keywords: Multiple hypotheses, Optimal statistical test, Error probability,
Neyman-Pearson Lemma.

1 . In t r o d u c t io n

Th e N e ym a n -P e a r s o n le m m a is t h e fo u n d a t io n o f t h e m a t h e m a t ic a l t h e o r y o f s t a t is t ic a l
h yp o t h e s is t e s t in g .

W e c a ll t h e s t a t is t ic a l h yp o t h e s is e a c h s u p p o s it io n s t a t e m e n t wh ic h m u s t b e ve r ī e d c o n -
c e r n in g t h e p r o b a b ilit y d is t r ib u t io n o f a n o b s e r va b le r a n d o m o b je c t . Th e t a s k o f s t a t is t ic ia n
is t o c o n s t r u c t a n a lg o r it h m ( t e s t ) fo r e ®e c t ive d e t e c t io n o f t h e h yp o t h e s is wh ic h is r e a liz e d .
Th e d e c is io n m u s t b e m a d e o n t h e b a s e o f ve c t o r o f r e s u lt s o f N in d e p e n d e n t id e n t ic a lly d is -

t r ib u t e d e xp e r im e n t s , c a lle d a s a m p le a n d d e n o t e d b y x
4
= ( x1; :::; xn; :::; xN ) , t h e e le m e n t s

o f X N , wh e r e X is t h e s p a c e o f p o s s ib le r e s u lt s o f e a c h e xp e r im e n t .
Th e p r in c ip le o f N e ym a n -P e a r s o n p la ys a c e n t r a l r o le in b o t h t h e t h e o r y a n d p r a c t ic e o f

s t a t is t ic s .
Th e r e e xis t s a va s t lit e r a t u r e wh e r e t h e t h e o r y o f t h e h yp o t h e s is t e s t in g a n d t h e N e ym a n -

P e a r s o n le m m a a r e e xp o u n d e d in d e t a il [1 ]{ [1 0 ]. Th e p a r a d ig m o f N e ym a n -P e a r s o n is fr e -
qu e n t ly u s e d in d i®e r e n t a p p lic a t io n s [1 1 ]{ [1 3 ]. B u t t h e m o s t p a r t o f t h e s e t e xt s is d e d ic a t e d
t o t h e c a s e o f t wo h yp o t h e s e s .

Th e p o s s ib ilit y o f e xt e n s io n o f Fu n d a m e n t a l L e m m a t o t h e c a s e o f m u lt ip le h yp o t h e s e s
is m e n t io n e d in [3 ]. S in c e t h e t e s t in g o f h yp o t h e s is is a c t u a l in a p p lic a t io n s we p r e s e n t o u r
ve r s io n o f t h e L e m m a fo r t h e c a s e o f t h r e e , o r m o r e h yp o t h e s e s .

Th e id e a o f t h is s t u d y wa s fo r m u la t e d in [4 ].

2 . P r o b le m S t a t e m e n t a n d R e s u lt Fo r m u la t io n

L e t P ( X ) b e t h e s p a c e o f a ll p r o b a b ilit y d is t r ib u t io n s ( P D s ) o n X . L e t X b e R V t a kin g
va lu e s in X wit h o n e o f M c o n t in u o u s P D s Gm 2 P ( X ) , m = 1 ; M. L e t t h e s a m p le x =

3 4



E. Haroutunian, P. Hakobyan 3 5

( x1; :::; xn; :::; xN ) , xn 2 X , n = 1 ; N , b e a ve c t o r o f r e s u lt s o f N in d e p e n d e n t o b s e r va t io n s
o f X.

B a s e d o n d a t a s a m p le a s t a t is t ic ia n m a ke s a d e c is io n wh ic h o f t h e p r o p o s e d h yp o t h e s e s
Hm : G = Gm, m = 1 ; M , is c o r r e c t .

Th e p r o c e d u r e o f d e c is io n m a kin g is a n o n -r a n d o m iz e d t e s t 'N ( x ) , it c a n b e d e ¯ n e d
b y d ivis io n o f t h e s a m p le s p a c e X N o n M d is jo in t s u b s e t s Am, m = 1 ; M. Th e s e t Am,
m = 1 ; M, c o n s is t s o f ve c t o r s x fo r wh ic h t h e h yp o t h e s is Hm is a d o p t e d .

W e s t u d y t h e p r o b a b ilit ie s o f t h e e r r o n e o u s a c c e p t a n c e o f h yp o t h e s is Hl p r o vid e d t h a t
Hm is t r u e

®ljm ( 'N )
4
= GNm ( Al ) =

X

x: x2A l
GNmx ) ; m; l = 1 ; M; m 6= l:

If t h e h yp o t h e s is Hm is t r u e , b u t it is n o t a c c e p t e d t h e n t h e p r o b a b ilit y o f e r r o r is t h e
fo llo win g :

®mjm ( 'N )
4
=

X

l:l 6=m
®ljm ( 'N ) = 1 ¡ GNm ( Am ) ; m = 1 ; M:

Fo r t h e g ive n p r e a s s ig n e d va lu e s 0 < ®¤1j1; ®
¤
2j2; ::::; ®

¤
M¡1jM ¡1 < 1 we c h o o s e n u m b e r s T1,

T2, ..., TM ¡1 a n d s e t s Am, m = 1 ; M, s u c h t h a t

A¤1 =
(
x : m in

Ã
G1 ( x )

G2 ( x )
; :::;

G1 ( x )

GM ( x)

!
> T1

)
; 1 ¡ GN1 ( A¤1 ) = ®¤1j1;

A¤2 = A¤1 \
(
x : m in

Ã
G2 ( x )

G3 ( x )
; :::;

G2 ( x )

GM ( x )

!
> T2

)
; 1 ¡ GN2 ( A¤2 ) = ®¤2j2;

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

A¤M¡1 = A¤1 \ A¤2 \ ::: \ A¤M ¡2 \
(
x :

GM¡1 ( x)

GM ( x )
> TM ¡1

)
; 1 ¡ GNM¡1 ( A¤M ¡1 ) = ®¤M¡1jM¡1;

a n d
A¤M = X N ¡ ( A¤1 [ A¤2 [ ::: [ A¤M ¡1 ) = A¤1 \ A¤2 \ ::: \ A¤M¡1:

Th e c o r r e s p o n d in g e r r o r p r o b a b ilit ie s a r e d e n o t e d b y

®¤ljm ( 'N ) ; m; l = 1 ; M ¡ 1 :

T heor em: The test determined by the sets A¤1, A¤2, ....., A¤M is optimal in the sense that,
for each other test de¯ned by the set B1, B2, ...., BM with the corresponding error probabilities
¯ljm; m; l = 1 ; M,

if ¯1j1 · ®¤1j1; then m a x( ¯1j2; ¯1j3; :::; ¯1jM ) ¸ m a x( ®¤1j2; ®¤1j3; ::::; ®¤1jM ) ;

if ¯2j2 · ®¤2j2; then m a x( ¯2j3; ¯2j4; :::; ¯2jM ) ¸ m a x( ®¤2j3; ®¤2j4; ::::; ®¤2jM ) ;
:::::::::::::::::::::::::::::::::::::::::::::::

and if ¯M ¡1jM¡1 · ®¤M¡1jM¡1; then ¯M ¡1jM ¸ ®¤M¡1jM :

Fo r s im p lic it y o f fo r m u la t io n s we p r e s e n t t h e p r o o f o f t h e Th e o r e m fo r M = 3 .



3 6 On Neyman-Pearson Principle in Multiple Hypotheses Testing

In t h a t c a s e fo r t h e g ive n va lu e s 0 < ®¤1j1; ®
¤
2j2 < 1 a n d c h o s e n n u m b e r s T1 a n d T2 s e t s

A¤m, m = 1 ; 3 , a r e t h e fo llo win g :

A¤1 =
(
x : m in

Ã
G1 ( x)

G2 ( x)
;

G1 ( x )

G3 ( x )

!
> T1

)
; 1 ¡ GN1 ( A¤1 ) = ®¤1j1;

A¤2 = A¤1 \
(
x :

G2 ( x )

G3 ( x )
> T2

)
; 1 ¡ GN2 ( A¤2 ) = ®¤2j2;

a n d
A¤3 = X N ¡ ( A¤1 [ A¤2 ) :

Th e c o r r e s p o n d in g e r r o r p r o b a b ilit ie s a r e d e n o t e d b y

®¤ljm ( 'N ) ; m; l = 1 ; 3 :

W e m u s t p r o ve t h a t t h e t e s t d e t e r m in e d b y t h e s e t s A¤1, A¤2 a n d A¤3 is o p t im a l in t h e s e n s e
t h a t , fo r e a c h o t h e r t e s t d e ¯ n e d b y t h e s e t s B1, B2 a n d B3 wit h t h e c o r r e s p o n d in g e r r o r
p r o b a b ilit ie s ¯ljm, m; l = 1 ; 3 , if ¯1j1 · ®¤1j1, t h e n m a x( ¯1j2; ¯1j3 ) ¸ m a x( ®¤1j2; ®¤1j3 ) , a n d if
¯2j2 · ®¤2j2 t h e n ¯2j3 ¸ ®¤2j3.

P r oof: L e t ©A ¤m a n d © B m b e t h e in d ic a t o r fu n c t io n s o f t h e d e c is io n r e g io n s A¤m a n d Bm.
Fo r a ll x = ( x1; x2; :::; xN ) 2 X N , t h e fo llo win g in e qu a lit y is c o r r e c t

( ©A ¤1 ( x ) ¡ ©B 1 ( x ) ) ( G1 ( x ) ¡ m a x( T1G2 ( x ) ; T1G3 ( x) ) ) ¸ 0 :

Mu lt ip lyin g a n d t h e n s u m m in g o ve r X N we o b t a in

©A ¤1 ( x ) G1 ( x) ¡ ©A ¤1 ( x) m a x( T1G2 ( x) ; T1G3 ( x) )

¡©B 1 ( x) G1 ( x ) + ©B 1 ( x) m a x( T1G2 ( x) ; T1G3 ( x ) ) ¸ 0 ;
X

x: x2X N

h
© A ¤1 ( x) G1 ( x) ¡ ©A ¤1 ( x) m a x( T1G2 ( x ) ; T1G3 ( x ) )

¡© B 1 ( x ) G1 ( x) + © B 1 ( x) m a x( T G2 ( x) ; T1G3 ( x) ) ] ¸ 0 ;
X

x: x2A ¤1
[G1 ( x ) ¡ T1 m a x( G2 ( x ) ; G3 ( x) ) ] ¡

X

x: x2B 1
[G1 ( x) ¡ T1 m a x( G2 ( x) ; G3 ( x ) ) ] ¸ 0 ;

1 ¡ ®¤1j1 ¡ T1 m a x( ®¤1j2; ®¤1j3 ) ¡ ( 1 ¡ ¯1j1 ) + T1 m a x( ¯1j2; ¯1j3 ) ¸ 0 ;
¡¯1j1 + ®¤1j1 · T1[¡ m a x( ®¤1j2; ®¤1j3 ) + m a x( ¯1j2; ¯1j3 ) ]:

W e s e e n o w t h a t fr o m ¯1j1 · ®¤1j1, it fo llo ws t h a t m a x( ¯1j2; ¯1j3 ) ¸ m a x( ®¤1j2; ®¤1j3 ) .
Th e p r o o f o f t h e o t h e r c a s e is s im ila r . Th e fo llo win g in e qu a lit y t a ke s p la c e fo r a ll x 2 X N

( ©A ¤2 ( x) ¡ ©B 2 ( x) ) ( G2 ( x) ¡ T2G3 ( x) ) ¸ 0 :

Mu lt ip lyin g a n d a ft e r t h a t s u m m in g o ve r X N we g e t

© A ¤2 ( x) G2 ( x ) ¡ ©A ¤2 ( x ) T2G3 ( x ) ¡ ©B 2 ( x ) G2 ( x ) + ©B 2 ( x ) T2G3 ( x ) ¸ 0 ;
X

x: x2X N

h
©A ¤

2
( x) G2 ( x ) ¡ ©A ¤

2
( x ) T2G3 ( x ) ) ¡ ©B 2 ( x ) G2 ( x ) + ©B 2 ( x ) T2G3 ( x )

i
¸ 0 ;



E. Haroutunian, P. Hakobyan 3 7

X

x: x2A ¤
2

[G2 ( x) ¡ T2G3 ( x) ] ¡
X

x: x2B 2
[G2 ( x ) ¡ T2G3 ( x ) ] ¸ 0 ;

1 ¡ ®¤2j2 ¡ T2®¤2j3 ¡ ( 1 ¡ ¯2j2 ) + T2¯2j3 ¸ 0 ;
¡¯2j2 + ®¤2j2 · T2 ( ¯¤2j3 ¡ ®¤2j3 ) :

It is c le a r t h a t if ¯2j2 · ®¤2j2, t h e n ¯2j3 ¸ ®¤2j3.
Th e t h e o r e m is p r o ve d .

3 . Co n c lu s io n

In t h is p a p e r we g e n e r a liz e d N e ym a n -P e a r s o n c r it e r io n o f o p t im a lit y fo r m a n y c o n t in u o u s
h yp o t h e s e s . W h e n d is t r ib u t io n s o f X a r e d is c r e t e t h e L e m m a c a n b e r e fo r m u la t e d wit h u s e
o f r a n d o m iz a t io n a s it is n o t e d in [3 ], [4 ] a n d [7 ].

B a ye s ia n t e s t in g wa s c o n s id e r e d fo r t h e c a s e o f t wo a n d m o r e h yp o t h e s e s in [4 ], [5 ].
It is d e s ir a b le h a ve in t e n t io n t o c o n s id e r m u lt yh yp o t h e s e s B a ye s ia n t e s t in g fo r t h e m o d e l
c o n s is t in g o f m a n y o b je c t s .

A c kn o wle d g e m e n t

Th is wo r k wa s s u p p o r t e d in p a r t b y S CS o f ME S o f R A u n d e r Th e m a t ic P r o g r a m N o S CS
1 3 { 1 A 2 9 5 .

Refer ences

[1 ] J. N e ym a n a n d E . S . P e a r s o n , \ On t h e p r o b le m o f t h e m o s t e ± c ie n t t e s t s o f s t a t is t ic a l
h yp o t h e s e s " , P h il. Tr a n s . R o y. S o c . L o n d o n , S e r . A , 2 3 1 , p p . 2 8 9 -3 3 7 , 1 9 3 3 .

[2 ] J. N e ym a n , F irst Course in P robability and Statistics, H o lt , R in e h a r t a n d W in s t o n ,
N e w Y o r k, 1 9 5 0 .

[3 ] E . L . L e h m a n a n d J.P . R o m a n o , Testing statistical hypotheses, Th ir d E d it io n . S p r in g e r ,
N e w Y o r k, 2 0 0 5 .

[4 ] A . A . B o r o vko v, M athematical Statistics, in R u s s ia n , N a u ka , Mo s c o w, 1 9 9 7 .

[5 ] H . L . V a n Tr e e s , D etection, E stimation and M odulation Theory, P a r t 1 . N e w Y o r k:
W ile y, 1 9 6 8 .

[6 ] M. H . D e Gr o o t , P robability and Statistics, 2 n d e d ., R e a d in g , MA , A d d is o n -W e s le y,
1 9 8 6 .

[7 ] M. G. K e n d a ll a n d A . S t u a r t , The Advanced Theory of Statistics, 2 , In fe r e n c e a n d
r e la t io n s h ip , Th ir d e d it io n . H a fn e r p u b lis h in g c o m p a n y, L o n d o n , 1 9 6 1 .

[8 ] A . K . B e r a , \ H yp o t h e s is t e s t in g in t h e 2 0 -t h c e n t u r y wit h a s p e c ia l r e fe r e n c e t o t e s t in g
wit h m is s p e c i¯ e d m o d e ls " , In : \ S t a t is t ic s fo r t h e 2 1 -s t c e n t u r y. Me t h o d o lo g ie s fo r
a p p lic a t io n s o f t h e Fit ir e " , Ma r c e l D e kke r , In c ., N e w Y o r k, B a s e l, p p . 3 3 -9 2 , 2 0 0 0 .

[9 ] I. Cs is z ¶a r a n d J. K Äo r n e r , Information Theory: Coding Theorems for D iscrete M emo-
ryless Systems, A c a d e m ic P r e s s , N e w Y o r k, 1 9 8 1 , ( R u s s ia n t r a n s la t io n , Mir , Mu s c o w,
1 9 8 5 ) .



3 8 On Neyman-Pearson Principle in Multiple Hypotheses Testing

[1 0 ] T. M. Co ve r a n d J. A . Th o m a s , E lements of Information Theory, S e c o n d E d it io n .
W ile y, N e w Y o r k, 2 0 0 6 .

[1 1 ] E . L e vit a n a n d N . Me r h a v, \ A c o m p e t it ive N e ym a n -P e a r s o n a p p r o a c h t o u n ive r s a l
h yp o t h e s is t e s t in g wit h a p p lic a t io n s " , IE E E Transactions on Information Theory, vo l.
4 8 , n o . 8 , p p . 2 2 1 5 { 2 2 2 9 , 2 0 0 2 .

[1 2 ] P . Mo u lin , \ A N e ym a n -P e a r s o n a p p r o a c h t o u n ive r s a l e r a s u r e a n d lis t d e c o d in g " , ISIT,
To r o n t o , Ca n a d a , Ju ly 6 -1 1 , 2 0 0 8 .

[1 3 ] P .-N . Ch e n , \ Ge n e r a l fo r m u la s fo r t h e N e ym a n -P e a r s o n t yp e -II e r r o r e xp o n e n t s u b je c t
t o ¯ xe d a n d e xp o n e n t ia l t yp e -I e r r o r b o u n d s " , IE E E Transactions on Information
Theory, vo l. 4 2 , n o . 1 , p p . 3 1 6 { 3 2 3 , 1 9 9 6 .

[1 4 ] C. C. L e a n g a n d D . H . Jo h n s o n , \ On t h e a s ym p t o t ic s o f M-h yp o t h e s is B a ye s ia n d e -
t e c t io n " , IE E E Transactions on Information Theory, vo l. 4 3 , n o . 1 , p p . 2 8 0 { 2 8 2 , 1 9 9 7 .

[1 5 ] E . A . H a r o u t u n ia n , \ N e ym a n -P e a r s o n p r in c ip le fo r m o r e t h a n t wo h yp o t h e s e s " , Ab-
stracts of Armenian M athematical Union Annual Session D edicated to 90 Anniversary
of R afael Alexandrian, Y e r e va n , p p .4 9 { 5 0 , 2 0 1 3 .

Submitted 22.08.2013, accepted 10.10.2013.

Ü»ÛÙ³Ý-äÇñëáÝÇ ëϽµáõÝùÁ µ³½Ù³ÏÇ í³ñϳÍÝ»ñÇ
ï»ëï³íáñÙ³Ý í»ñ³µ»ñÛ³É

º. гñáõÃÛáõÝÛ³Ý ¨ ö. гÏáµÛ³Ý

²Ù÷á÷áõÙ

²Ûë ³ß˳ï³ÝùáõÙ Ý»ñϳ۳óí³Í ¿ Ü»ÛÙ³Ý-äÇñëáÝÇ ¹³ë³Ï³Ý É»ÙÙ³ÛÇ ÁݹѳÝñ³óáõÙÁ
»ñÏáõëÇó ³í»ÉÇ í³ñϳÍÝ»ñÇ í»ñ³µ»ñÛ³É:

Î ïðèíöèïå Íåéìàíà-Ïèðñîíà ïðè ïðîâåðêå
ìíîãèõ ãèïîòåç

Å. Àðóòþíÿí è Ï. Àêîïÿí

Àííîòàöèÿ

Öåëü íàñòîÿùåé ñòàòüè ïðåäñòàâèòü íîâîå îáîáùåíèå êëàññè÷åñêîé Ëåììû
Íåéìàíà-Ïèðñîíà äëÿ áîëåå ÷åì äâóõ ãèïîòåç.