5_wu.dvi


EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS

Vol. 5, No. 3, 2012, 333-356

ISSN 1307-5543 – www.ejpam.com

Data Fusion Using Weighted Likelihood

Pengfei Guo, Xiaogang Wang, Yuehua Wu∗

Department of Mathematics and Statistics, York University, Toronto, Canada

Abstract. This article proposes to perform data fusion by using an adaptive weighted likelihood func-

tion when data sets are available from related populations. The main objective of data fusion is to

integrate information from different sources to improve the quality of inference when the sample size

from the target population is small or moderate. The weighted likelihood function is employed simply

as an instrument to facilitate the data fusion process. The weighted likelihood method has information-

theoretic justification and embraces the widely used classical likelihood method which utilizes only on

the data set from the target population. The degree of information integration in the proposed data

fusion process is determined by the likelihood weights which should be chosen in a reasonable and

adaptive way. The major challenge in the proposed data fusion process is then to choose likelihood

weights adaptively and effectively when the deterministic relationships among all related parameters

are unknown. We propose adaptive likelihood weights based on the estimated likelihood ratio. We

show that the data fusion involving all relevant data sets could significantly improve the mean squared

error (MSE) of the classical maximum likelihood estimator which only uses data set from the target

population. It also increases the power for hypothesis testing. The proposed estimator is shown to be

consistent and asymptotically normally distributed in the framework of generalized linear models. The

advantage of the proposed weighted likelihood estimator for linear models is illustrated numerically

by a simulation study. A real data example is also provided.

2010 Mathematics Subject Classifications: 62F12, 62J05, 62J12

Key Words and Phrases: Cross-validation, Nonparametric regression, Relative likelihood ratio, Semi-

parametric estimation, Weighted likelihood

1. Introduction

In clinical trials for medical research, a common problem that often arises is how to ef-

ficiently estimate the parameter of interest when the sample size from the population of in-

ferential interest is small due to the cost or other limitations associated with the experiment.

When the sample size from the population of inferential interest is small and observations

from related studies are available, one important question is whether there is any merit to

∗Corresponding author.

Email addresses: pguo�mathstat.yorku.
a (P. Guo), stevenw�mathstat.yorku.
a (S. Wang),

wuyh�mathstat.yorku.
a (Y. Wu)

http://www.ejpam.com 333 c© 2012 EJPAM All rights reserved.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 334

combine information from populations that are known to be different than the population of

inferential interest.

The classical likelihood method would concentrate solely on the sample from the the

target population without incorporating other relevant information. The advantage of doing

so is to avoid possible bias or contamination of the data sampled directly from the target

population. The maximum likelihood estimator (MLE) is well known to enjoy asymptotic

properties such as consistency and asymptotic normality. It is one of the most widely used

method in statistical inference. However, the asymptotic properties are of little help when

the sample size is small or very moderate. The MLE could provide quite misleading results

due to a insufficient sample size. By contrast, the Bayesian method can effectively combine

information when the parameters from these populations are assumed to be random variables

from a hyper-distribution. It has many advantages since prior information can be formally

incorporated and the inferences are conditional on all the data. Bayesian inference might

be computationally intensive for non- trivial cases. It is well known, however, the powerful

Monte Carlo Markov Chain method could be computationally intensive and challenging for

the high dimensional case when a conjugate prior can not be assumed.

For exploratory purposes, we propose to integrate all available information through a

data fusion process based on an adaptive weighted likelihood function. The weighted likeli-

hood function that we employ can be very adaptive and easy to implement. Since there are

many weighted likelihoods proposed for various purposes, it is necessary to provide an brief

overview and avoid any possible confusion later on. [4] introduced the idea of local likeli-

hood to derive local inference. [2] defined her version of local likelihood in the context of

non-parametric regression. [17] presented the general form of the local likelihood. [9] and

[10] proposed their relevant weighted likelihood function to combine information in which

the weights are very general. [12] proposed their adaptive weights when the time trend is

present. [11] reviewed most related weighted likelihood methods proposed in the literature.

All the aforementioned weighted likelihood methods focus on a setting in which the number

of populations goes to infinity. In this article, however,we focus on situations in which the

number of populations is actually fixed. For example, one might be interested in the efficacy

of one particular treatment when the results of the current and several historical clinical trials

are available.

Extending the relevant weighted likelihood by [9], [21] proposed their version of the WL

when the number of populations is fixed. We now briefly their weighted likelihood function

as follows. Suppose that we observe independent random response vectors Y 1, . . . , Y m with

probability density functions f (.;θ1), . . . , f (.;θm), where Y i = (Yi1, . . . , Yini
)T, i = 1,2, · · · , m.

Further suppose that only population 1, in particular θ1, an unknown vector of parameters, is

of inferential interest. Given data, Y 1 = y1, the classical likelihood would be

L1(y1,θ1) =

n1∏

j=1

f (y1 j;θ1).

When the parameters θ2, . . . ,θm are thought to interconnected with θ1, for given



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 335

y = (y1, y2, · · · , ym), the weighted likelihood (WL) for θ1 is defined as

WL(y;θ1) =

m∏

i=1

ni∏

j=1

f (yi j;θ1)
λi ,

where λ = (λ1, . . . ,λm), the “weights vector”, must be specified. We emphasize that the

parameters from the related populations, θ2, . . . ,θm, which are unknown, do not appear in

the WL, since the inferential interest focuses on θ1 of the first population.

The WL estimator (WLE) for θ1, say θ̃1, is defined as the maximizer of the objective

function WL(y;θ1):

θ̃1 = arg sup
θ1∈Θ

WL(y;θ1).

In order for the weighted likelihood to be effective, the weights must be chosen adaptively so

that all information obtained from related population must be evaluated to determine their

relevance. [7] proposed a cross-validatory procedure for weights selection and showed that

the resulting WLE is consistent and asymptotically normal. However, the cross-validatory

approach is challenged with the following problems: (a) it is computationally intensive if the

WLE does not have an analytical form, and (b) numerical performances could be unstable

when the sample sizes are unequal or the sample sizes are very small. Thus, we propose

a simple and effective method to choose the likelihood weights adaptively. This method is

better than the cross-validated weights proposed by [7] in that its computation is robust and

implementation is straightforward. More importantly, it avoids all the numerical problems

with the cross-validated weights when the sample sizes are small or unequal. We derive the

consistency and asymptotic distribution of the resulting WLE.

The article is organized as follows. In Section 2, we develop a WL procedure. The asymp-

totic properties of the WLE for generalized linear models using the proposed adaptive weights

are presented in Section 3. Section 4 presents the results from simulation experiments that

illustrate the numerical performance of our method. Section 5 shows how to obtain the em-

pirical distribution of the WLE by bootstrap. Section 6 analyzes a real data set from by using

the proposed approach. Section 7 provides conclusions and discussions.

2. The Weighted Likelihood and Adaptive Weights

2.1. The Weighted Likelihood

We assume the existence of the m population density functions are unknown and play

purely conceptual roles. More specifically, assume σ-finite probability spaces

(Ω,F ,µi), i = 1,2, . . . , m, with probability measures µi ’s that are absolutely continuous with

respect to one another. The existence of a σ-finite measure ν that dominates the µi ’s then

follows. We take the fi to be the Radon-Nikodym derivatives of µi with respect to ν for

i = 1,2, . . . , m.

Suppose that we observe independent random response vectors Y 1, . . . , Y m with probabil-

ity density functions f (.;θ1), . . . , f (.;θm), where Y i = (Yi1, . . . , Yini
)T, i = 1,2, · · · , m. Further



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 336

suppose that only population 1, in particular θ1, an unknown vector of parameters, is of

inferential interest. Given data, Y1 = y1, the classical likelihood of θ1 is defined by

L1(y1;θ1) =

n1∏

j=1

f (y1 j;θ1).

Similarly, we can define the likelihood for each θi as follows:

Li(y i ,θ1) =

ni∏

j=1

f (yi j;θi).

where i = 2, · · · , m.

If all parameters are not related in any way, the correct likelihood for θ1 should be

L1(y1;θ1). However, L1(y1;θ1) would not be the correct likelihood if there exist functional

relationship among all parameters. To be more specific, assume that

θ2 = g1(θ1), . . . ,θm = gm(θ1), where gi are measurable functions of θ1. Furthermore, if all

observations from different populations are independent of each other, the likelihood of θ1

involving all samples should be

L̃(y1, · · · , ym;θ1) = L1(y1,θi) ×
m∏

i=2

Li(y i, gi(θ1)).

We observe that all parameters, θ2, . . . ,θm disappeared from the above likelihood since they

are replaced by g2(θ1), . . . , gm(θm) respectively. We see that all data sets are integrated into

one likelihood function that we call fusion likelihood.

When the functional relationship, gi, are indeed known, we can derive the estimating

equation by consider the following:

log L̃(y1, . . . , ym;θ1)

∂ θ1

=
log L1(y1;θ1)

∂ θ1

+

m∑

i=2

log Li(y i;θi)

∂ θi

gi(θ1)

∂ θ1

=
log L1(y1;θ1)

∂ θ1

+

m∑

i=2

wi(θ1)
log Li(y i;θi)

∂ θi

.

where wi(θ1) = ∂ gi(θ1)/∂ θ1, i = 2, · · · , m. By setting the above function to be zero, we

obtain an estimating equation for θ1 which is based on all available data.

In practice, however, the functional form of gi ’s are not known. Therefore, the analytical

forms of gi ’s are simply not available. When facing such a difficulty, one could abandon the

data fusion process and proceed with L1 by severing the connections among all parameters.

Although this would avoid any possible bias, this approach would clearly lose some informa-

tion. The loss of information should not be of great concern if the sample size from the target

population is large enough to ensure a valid and effective inference. The benefit of integrating

information from other sources is clearly negligible. When the sample from the target popu-

lation is very small or moderate, the related data sets discarded as the result of avoiding bias

could contain very valuable information.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 337

The alternative is try to incorporate information from other sources in a meaningful way.

This must be done in a reasonable and careful fashion without the knowledge of the link func-

tions gi, i = 2, · · · , m. First, we must adopt a meaningful measure to evaluate the discrepancy

among the probability density distributions when the parameters are known. Second, we

must tackle the problem without the knowledge of the true values of the parameters. To re-

solve the first challenge, we propose to use some general measure for information discrepancy

to characterize the connections among all parameters. Entropy or relative entropy has been

widely used in information theory. For density functions, g1(x) and g2(x), with respect to a

σ−finite measure ν , the relative entropy, also called Kullback-Leibler divergence, is defined

as:

K L(g1, g2) = E1

�
log

g1(X )

g2(X )

�
=

∫
log

g1(x)

g2(x)
g1(x)dν(x). (1)

In this expression, log(g1(x)/g2(x)) is defined as +∞, if g1(x)> 0 and g2(x) = 0. Therefore

the expectation could be +∞. Although log(g1(x)/g2(x)) is defined as −∞ when g1(x) = 0

and g2(x) > 0, the integrand, log(g1(x)/g2(x))g1(x) is defined as zero in this case. It is

widely used in information theory. Detailed discussions and application of the entropy in

information theory can be found in [19]. Some theoretical properties of the entropy can

be found in [14]. In particular, the relative entropy is not symmetric and therefore not a

distance. The relative entropy, also known as Kullback-Leibler divergence, is also called the

entropy loss. [20] introduce it as a performance criterion in estimating the multinormal

variance-covariance matrix. [15] shows the entropy is a loss function in a Bayesian frame-

work. [18] provides detailed discussions on the relative entropy including the connection

between Fisher information and relative entropy. [13] has shown that the classical maximum

likelihood principle can be considered a method of asymptotic realization of an optimum

estimate with respect to the relative entropy.

Since the metric of the manifold that connects all the parameters can not be specified, we

now define the neighborhood enveloping by [17]. For any fixed value of θ1, the parameter

of the population of inferential interest, we can define a neighborhood by using the relative

entropy as

N f1
(ε) = ∪θ∈Θ{ f (θ) : I( f (θ), f1)≤ ε}, (2)

where f1 = f (x ,θ1) and ε≥ 0.

Assume that the density functions, f1, . . . , fm ∈ V , are all assumed to be continuous where

V is a reflexive Banach space. Although V can be quite arbitrary, we take V = Lp = Lp(Ω,ν).

It is known that the Lp spaces (1 < p < ∞) are reflexive but that L1 is not [see 16, for

example]. For i = 2, . . . , m, we define

Ei = {g ∈ Lp : ||g − fi ||p < Ci,

∫
fi(x) log

g(x)

fi(x)
dν(x)≤ ai,

∫
g(x)dν(x) = 1, g(x)> 0}, (3)

where ai ≥ 0 and Ci, i = 2,3, . . . , m, are constants. We then define the interception of these

individual enveloping neighborhood.

E = ∩m
i=2Ei. (4)



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 338

We remark that the set E will be bounded with respect to the Lp norm and non-empty if the

constraints are not too restrictive. The latter is assumed throughout.

Thus, for a given set of density functions, f1(x) being primary, we seek a probability

density function g ∈ E which minimizes I( f1, g) =
∫

f1(x) log
g(x)

f1(x)
dν(x) over all probability

densities, g, satisfying

I( fi , g) ≤ ai, i = 2, . . . , m, (5)

where ai, i = 2,3, . . . , m, are non-negative constants.

[6] showed that the the optimal solution takes the form of a mixing distribution:

g∗ =
m∑

i=1

λi fi(x), (6)

where g∗ is the optimal solution for optimization probelm described above.

Assuming that fi are members of one parametric family while the difference is due to

different value for the parameter, i.e. fi(x) = f (x ;θi). Without the knowledge of the exact

values for θi , [6] show that the weighted likelihood function:

WL(y;θ1) =

m∏

i=1

ni∏

j=1

f (yi j;θ1)
λi , (7)

is the correct device to used by following the argument of [13]. The weights are functions of

the λi ’s which measures the discrepancies among all parameters.

We remark that this classical likelihood is embraced by the proposed likelihood by setting

the weights to be zero for relevant samples. In addition, [5] showed that the estimator derived

from the weighted likelihood function can be viewed as an approximate Bayes decision by

following the same argument by [3]. The coefficients ai ’s, however, are not known since the

exact functional form of the connections among all parameters are not assumed to be known.

Therefore, the weights must be chosen adaptively to avoid introduction of significant bias into

the inference for the target population.

2.2. Adaptive Weights Based on Likelihood Ratio

Since the weighted likelihood is the chosen instrument for the data fusion process, the

likelihood weights play a central role in integrating all relevant information in a meaningful

way. As we have seen in previous section, the likelihood weights should be chosen to reflect

the true relevance between a relevant sample and the target population. Without any prior

knowledge, the evidence expressed in the relevant sample should be the only source for eval-

uation and determination of a set of appropriate weights. The fundamental question is how

to measure the discrepancy between to two populations that could be related.

In a likelihood ratio context, a likelihood-ratio test is a statistical test for making a decision

between two hypotheses based on the value of this ratio.

Since the parameters of these two populations are not known, therefore we plug in the

MLE estimates. When the ratio measures the resemblance between the two values. similarly



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 339

to the hypothesis test, it implies that the two parameter values are too different in the context

of likelihood ratio. Following the same line of argument, we propose to use the likelihood

ratio as a measure of discrepancy. Since the true values of the parameters are not known, it is

natural to use estimated likelihood ratio with the parameter values replaced by corresponding

MLE estimates.

Given a parametric density function f1 and a sample Y 1 = (Y11, Y12, . . . , Y1n1
), the classical

likelihood ratio for testing a null hypothesis is based on the following

LR =

supΩH

n1∏
j=1

f1(Y1 j;θ1)

supΩ

n1∏
j=1

f1(Y1 j;θ1)

,

where ΩH is a subspace in the parameter space under the null hypothesis H.

Assume that the likelihood function attains an unique maximum in ΩH and Ω respectively.

Denote the maximizers as bθH
1 and bθ1 respectively. The likelihood ratio (LR) can then be

rewritten as

LR =

n1∏
j=1

f1(Y1 j;
bθH

1 )

n1∏
j=1

f1(Y1 j;
bθ1)

.

Therefore, the validity of the proposed hypothesis is evaluated according to the likelihood

ratio.

By using a similar idea, we propose to choose likelihood weights by using the estimated

pairwise likelihood ratio. For simplicity, let m = 2, i.e., there is only one related population

together with the target population. Let bθ2 denote the estimate derived from the second

sample. We consider the estimated pairwise likelihood ratio (PLR) as follows:

γi = P LRi =

n1∏
j=1

f1(Y1 j;
bθi)

n1∏
j=1

f1(Y1 j; bθ1)

, i = 1,2.

It then follows that γ1 equals to 1. Furthermore, we have γi ≤ 1, i = 2, . . . , m by the definition

of the MLE.

The likelihood weights λ1 and λ2 will then be chosen as

λ1 =
1

1+ γ2

, λ2 =
γ2

1+ γ2

.

For the simple linear models presented before, a simplification yields that the WLE of β1 takes

the following form:
bβWLE

1 = w1
bβMLE

1 +w2
bβMLE

2 ,



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 340

where

w1 =

n1∑
j=1

x2
1 j

n1∑
j=1

x2
1 j
+ γ2

n2∑
k=1

x2
2k

and w2 =

γ2

n2∑
k=1

x2
2k

n1∑
j=1

x2
1 j
+ γ2

n2∑
k=1

x2
2k

.

To further illustrate the behavior of the proposed likelihood weights, we revisit the simple

linear models given by Equation (1). It then follows that

γ2 = exp



−

1

2σ2
1

n1∑

j=1

(y1 j − bβMLE

2 x1 j)
2 +

1

2σ2
1

n1∑

j=1

(y1 j − bβMLE

1 x1 j)
2





= exp



−(bβ

MLE

1 − bβMLE

2 )
2

n1∑

j=1

x2
1 j/(2σ

2
1)



 .

Since λ2 = 1/(1+ γ2), we then have

λ2 =
1

1+ exp

(
(bβMLE

1
− bβMLE

2
)2

n1∑
j=1

x2
1 j
/(2σ2

1
)

) .

Thus the proposed likelihood weight for the second sample is a function of the estimated dif-

ference between the regression coefficients, the variance of the error terms and the “variance”

of the covariate. For a fixed design matrix, a large difference between bβMLE

1 and bβMLE

2 will re-

duce the assigned importance to the second sample. On the other hand, a large variance in

the first model would result in a bigger value for the weight assigned to the second sample.

2.3. Estimation with Fixed Likelihood Weights

In this section we develop the WL procedure for the linear models as an illustration. For

simplicity, we assume that m = 2 in Sections 2.1 and 2.2. Extension to the case of m > 2

is straightforward. Assume that data {x1 j , y1 j; j = 1, . . . , n1} and {x2k, y2k; k = 1, . . . , n2} are

generated from the following models:

y1 j = β1 x1 j + ε j , ε j ∼ N(0,σ2
1), j = 1, . . . , n1,

y2k = β2 x2 j + ek; ek ∼ N(0,σ2
2), k = 1, . . . , n2,

(8)

where the {x1 j , j = 1, . . . , n1; x2 j , k = 1, . . . , n2} are fixed. Assume that {ε j}’s and ek’s are i.i.d.

respectively. The error terms from these two models are assumed to be independent as well.

For the purpose of our demonstration we assume σ1 and σ2 are known although that would

rarely be the case in practice. Denote y1 = (y11, y12, . . . , y1n1
)T, and y2 = (y21, y22, . . . , y2n2

)T.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 341

Suppose that the parameter, β1, is of primary inferential interest. The parameter β2 is

thought or expected to be not too different from β1 due to the “similarity” of the two experi-

ments. Note that a bivariate distribution is not assumed in the above model.

Assuming marginal normality, the marginal likelihoods for β1 and β2 are

L1(y1;β1)∝
n1∏

j=1

exp

¨
− (y1 j − β1 x1 j)

2

2 σ2
1

«
,

and

L2(y2;β2)∝
n2∏

k=1

exp

¨
− (y2k − β2 x2k)

2

2 σ2
2

«
.

A direct calculation deduces the MLE of β1 and β2 as follows

bβMLE

1 =




n1∑

j=1

x2
1 j




−1
n1∑

j=1

x1 j y1 j , and bβMLE

2 =

 
n2∑

k=1

x2
2k

!−1 n2∑

k=1

x2k y2k.

If one knows that β1 and β2 are similar to each other according to past studies or expert

opinions, then it is reasonable to expect that this information might be used to yield a better

estimate of the parameter β1.

The WL for inference about β1 can be represented as:

WL(β1; y1, y2) = L
λ1

1 (y1;β1)L
λ2

2 (y2;β1), (9)

where λ1 and λ2 are weights selected according to the relevance of the likelihood to which

they attached. The non-negative requirement for the weights is not assumed in the formu-

lation although the optimum weights should be non-negative according to the assertion by

[6].

It is worth noting that L2(y2;β1) instead of L2(y2;β2) is used to define WL(β1) since β1 is

of our primary inferential interest at this stage and the marginal distributions of the elements

of the Y 2 are thought to resemble the marginal distributions of those of the Y1. Note that

the WL for θ1 depends on the distribution of the Y 1. However, it does not depend on the

distribution of Y 2. Notice that the joint distribution of the Y 1 and Y 2 does not appear in the

formulation of the WL for θ1 and no assumptions are made about it.

The WLE of β1 is obtained by maximizing the weighted likelihood function for given

weights λ1 and λ2. It follows from (9) that

log{WL(β1)} = λ1 log{L1(y1;β1)}+λ2 log{L2(y2;β1)}.

Note that

∂ log{WL(β1)}
∂ β1

=
λ1

2σ2
1

n1∑

j=1

x1 j(y1 j − β1 x1 j)+
λ2

2σ2
1

n2∑

k=1

x2k(y2k −β1 x2k).



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 342

A simplification yields the WLE of β1 as follows

bβWLE

1 = w1
bβMLE

1 +w2
bβMLE

2 . (10)

where

w1 =

λ1

n1∑
j=1

x2
1 j

λ1

n1∑
j=1

x2
1 j
+λ2

n2∑
k=1

x2
2k

and w2 =

λ2

n2∑
k=1

x2
2k

λ1

n1∑
j=1

x2
1 j
+λ2

n2∑
k=1

x2
2k

.

It can be seen that the WLE of β1 is a linear combination of bβMLE

1 and bβMLE

2 , under the over-

simplified model. The WLE of β1 coincides with the MLE of β1 obtained from the first if the

weight for the second marginal likelihood function is set to be zero. Therefore, the weights

w1 and w2 reflect the importance of bβMLE

1 and bβMLE

2 .

Let β0
1 and β0

2 be the true values of the parameters β1 and β2 respectively. If one knows

that |β0
1 −β0

2 | ≤ C , where C is a known constant according to past studies or expert opinions,

we then have the following theorem.

Theorem 1. Under the normality assumption with known variances, the WLE β̃WLE

1 takes the

form

β̃WLE

1 (w1, w2) = w1
bβMLE

1 +w2
bβMLE

2 ,

where w1 + w2 = 1,0 < w1 ≤ 1,0 ≤ w2 < 1. Furthermore, |β0
1 − β0

2 | ≤ C, C > 0, then

|E(β̃1− β0
1 )| ≤ w2 C. In addition,

max
w1,w2

MSE(β̃WLE

1 )< MSE(bβMLE

1 ) if and only if
M

M + 1
< w1 < 1,

where

M =

C2 +σ2
2

�
n2∑

k=1

x2
2k

�−1

−σ2
1

 
n1∑
j=1

x2
1 j

!−1

2σ2
1

 
n1∑
j=1

x2
1 j

!−1
.

In addition, Var(β̃WLE

1 )< Var(bβMLE

1 ) if maxw1,w2
MSE(β̃WLE

1 )< MSE(bβMLE

1 ).

Thus, for small value of C, the WLE could have smaller MSE and variance simultaneously

with the cost of small bias. The magnitude of the bias is controlled by the weight assigned to the

second population. The lower bound for the weight w1 is connected with the worst MSE of the

WLE instead of the optimal one.

However, this theorem does not provide any guidance on how to choose the optimum

weights since the bound C is rarely known in practice. Even though the exact value of the

bound is known or can be estimated fairly accurately, this theorem still could not be used

to choose the best set of weights. For example, suppose that there are two independent

experiments with exactly the same coefficients and design matrices. Therefore, we should

simply merge these two experiments. However, the above theorem merely suggests that the

lower bound is zero for w1 since M is zero in this case.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 343

3. Asymptotic Properties

In this section we investigate the asymptotic properties of the WLE in the framework

of generalized linear models (GLM) for simple presentation. The GLMs have the following

structure. The sample Y = (Y1, . . . , Yn) ∈ ℜn has independent components and Yi has the

p.d.f.

exp

�
ηi yi − ζ(ηi)

φi

�
h(yi,φi), . . . , n,

w.r.t. a σ-finite measure ν , where ηi and φi are unknown, φi > 0,

ηi ∈ Ξ = {η : 0<

∫
h(y,φ)eηy/φdν(y)<∞} ⊂ ℜ

for all i, ζ and h are known functions, and ζ′′(η) > 0 is assumed for all η ∈ Ξ◦, the inferior of

Ξ. Note that the p.d.f. belongs to an exponential family if φ is known. As a consequence, for

i = 1, . . . , n,

E(Yi) = ζ
′(ηi)

△
= µ(ηi),

Var(Yi) = φζ
′′(ηi).

It is assumed that ηi is related to X i, the ith value of a p-vector covariates, through

g(µ(ηi)) = β
T X i, i = 1, . . . , n.

In GLM, β is the parameter of interest and φi ’s are considered to be nuisance parameters.

The range of β is assumed to be B = {β : (g ◦ µ)−1(β T x) ∈ Ξ◦∀x ∈ X}, where X is the

range of X i ’s. It is often assumed that

φi = φ/t i , i = 1, . . . , n,

with unknown φ > 0 and known positive t i ’s.

Let θ = (β ,φ) and ψ= (g ◦ µ)−1, then the log-likelihood function is

log l(θ ) =

n∑

i=1

�
log h(yi,φ/t i) +

ψ(β T X i)yi − ζ(ψ(β T X i))

φ/t i

�

and the score function for β is

sn(β) =
∂ log l(θ )

∂ β
=

1

φ

n∑

i=1

{[yi −µ(ψ(β T X i))]ψ
′(β T X i))t iX i}.

Let

Mn(β) =

n∑

i=1

[ψ′(β T X i)]
2ζ′′(ψ(β T X i))t iX iX

T
i .



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 344

Then the Fisher information evaluated at β is

In(β) = Var

�
∂ log l(θ )

∂ β

�
= Mn(β)/φ.

We now present the asymptotic properties for the relevant weighted likelihood estimator

and the hypothesis testing based on this estimator. We use double index as the subscript of

observations and establish the following notations.

• β0: The true parameter value of main group.

• y1 j , j = 1, . . . , n1 are the observations from main group.

• y2 j , j = 1, . . . , n2 are the observations from contamination group.

• wl(β) =
∑2

i=1

∑ni

j=1
wi j log f (yi j | x i,β) =

∑2

i=1

∑ni

j=1
wi j li j(β).

• l(β) =
∑2

i=1

∑ni

j=1
log f (yi j | x i,β) =

∑2

i=1

∑ni

j=1
li j(β).

Besides the regularity conditions and assumptions given in [1], we need the following

additional assumptions for the present problem.

(A1) The information matrix of main group In1
(β0) is positive definite, and the eigenvalues

of n−r
1

In1
(β0) are bounded, for some r > 0, so are the eigenvalues of nr

1 I−1
n1
(β0). That

is, there exist constants c1 and c2, such that

c1 ≤ λminnr
1I−1

n1
(β0)≤ λmaxnr

1I−1
n1
(β0)≤ c2.

(A2) There exists a neighborhood of β0, O, such that for all β ∈ O and y2 j , j = 1, . . . , n2,

E

(
sup

β∈O,1≤ j≤n2

�����log
f (y2 j ,β)

f (y2 j ,β0)

�����

)
≤ K ,

where K is a constant.

(A3) The proportion of contamination converges to 0 in the order of n−1/2, i.e., εn = o(n−1/2).

(A4) The criteria to choose weights guarantees that the weights are consistent. Assume the

first n1 observations are from main group and the rest are from contamination group.

Then as n→∞ and εn→ 0,

w = (w1, . . . , wn)
T → v = (v1, . . . , vn) = (1, . . . , 1︸ ︷︷ ︸

n1

, 0, . . . , 0︸ ︷︷ ︸
n2

)T .

Furthermore, maxi |wi − vi| = o(n−1/2).

We now provide the asymptotic results in Theorems 2-3 below.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 345

Theorem 2. Under the regularity conditions specified in [1] and assumptions (A1)–(A4), there

is a sequence of estimators {β̂n} such that

P(sn(β̂n) = 0)→ 1

and β̂n→ β0 in probability.

Proof. In order to present the proof clearly, we use double index as the subscript of obser-

vations. Then we need to specify the following notations.

• β0: The true parameter of main group.

• γ0: The true parameter of second group.

• y1 j , j = 1, . . . , n1 are the observations from main group.

• y2 j , j = 1, . . . , n2 are the observations from second group.

• wl(β) =
∑2

i=1

∑ni

j=1
wi j log f (yi j |x i ,β) =

∑2

i=1

∑ni

j=1
wi j li j(β).

• l(β) =
∑2

i=1

∑ni

j=1 log f (yi j |x i ,β) =
∑2

i=1

∑ni

j=1 li j(β).

Since there are two populations, we distinguish the score functions, Fisher information, design

matrices in the following way. For main group, we denote them as sn1
(β), In1

(β) and X ; and

for contamination group, we have sn2
(β), In2

(β) and Z . For simplicity, We use Mn1
to denote

Mn1
(β0).

Define the neighborhood of β0, Nn1
(δ), δ > 0, as

Nn1
(δ) = {β : ‖In1

(β0)
1/2(β −β0)‖ ≤ δ)},

where In1
(β0) is the Fisher information of the main group data evaluated at true value β0.

Let ∂ Nn1
(δ) be the boundary of Nn1

(δ). To prove β
p→ β0, it is sufficient to show for any

η > 0, there exist δ > 0 and N > 0, such that for n≥ N and all β ∈ ∂ Nn1
(δ), we have

P(wl(β)−wl(β0)< 0 for all β ∈ ∂ Nn1
(δ))> 1−η.

We start the proof by partitioning wl(β)−wl(β0).

wl(β)−wl(β0)

=

n1∑

j=1

�
l1 j(β)− l1 j(β0)

�
−

n1∑

j=1

(1−w1 j)
�

l1 j(β)− l1 j(β0)
�
+

n2∑

j=1

w2 j

�
l2 j(β)− l2 j(β0)

�

△
= A− B + C .

In [1], it is shown that there exist δ > 0 and N such that for n≥ N and all β ∈ ∂ Nn1
(δ),

P(A< 0)> 1−η.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 346

Then we need to show that B and C are dominated by A, which will follows from B and C are

op(1) since it is shown that A is Op(1) in [1].

In the following, we show that B is dominated by A. Define s1−w
n1
(β) as

s1−w
n1
(β) =

∂

∂ β

n1∑

j=1

(1−w1 j)l1 j(β)

=

�
∂ log f (y11,β)

∂β
, . . . ,

∂ log f (y1n1
,β)

∂ β

�



1−w11
...

1−w1n1




△
=

�
U1, . . . ,Un1

�
W1

△
= S1W1.

Let sn1
(β) be the score function for the main group, which can be similarly written as

sn1
(β) = S11n1

,

where 1n denotes the length-n vector with all element equal to 1. Then by taking Taylor

expansion of B, we have

B = (β −β0)
T s1−w

n1
(β0) +

1

2
(β −β0)

T∇s1−w
n1
(γ∗)(β −β0),

where γ∗ ∈ Nn1
(δ). In the following, we will show that both these two terms are op(1).

To show (β −β0)
T s1−w

n1
(β0) = op(1), we need to show for any ǫ > 0,

lim
n→∞ P(‖(β −β0)

T s1−w
n1
(β0)‖< ǫ) = 1

By Markov inequality, we have

P(‖(β −β0)
T s1−w

n1
(β0)‖< ǫ)≥ 1− (1

ǫ
)2E‖(β −β0)

T s1−w
n1
(β0)‖2

and so it is sufficient to show E‖(β −β0)
T s1−w

n1
(β0)‖2→ 0.

Let λ= In1
(β0)

1/2(β −β0)/δ, then ‖λ‖= 1 and

E‖(β −β0)
T s1−w

n1
(β0)‖2 = E‖(β −β0)

T S1W1‖2

= E‖δλT I−1/2
n1
(β0)S1W1‖2 ≤ δ2‖W1‖2E‖I−1/2

n1
(β0)S1‖2

= δ2‖W1‖2E
n

t r[ST
1 I−1

n1
(β0)S1]

o
= δ2‖W1‖2E





n1∑

j=1

U T
j I−1

n1
(β0)U j



 .



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 347

Because the eigenvalues of In1
(β0)/n

r
1 are bounded by assumption (A4), so are eigenvalues

of nr
1 I−1

n1
(β0). That is

c1 ≤ λmin(n
r
1I−1

n1
(β0))≤ λmax(n

r
1I−1

n1
(β0))≤ c2,

where c1 and c2 are constants. Because EUT
j Uk = 0, let Ip be the p × p identity matrix, we

have

E‖(β −β0)
T s1−w

n1
(β0)‖2 ≤ δ2‖W1‖2

1

n1

E





n1∑

j=1

U T
j (n

r
1I−1

n1
(β0))U j





≤ δ2‖W1‖2
1

n1

E





n1∑

j=1

UT
j [c2 Ip]U j





≤ δ2‖W1‖2
c2

nr
1c1

E
n

sn1
(β0)

T [nr
1I−1

n1
(β0)]sn1

(β0)
o

≤ δ2‖W1‖2
c2

nr
1
c1

Etr
n

sn1
(β0)

T[nr
1 I−1

n1
(β0)]sn1

(β0)
o
= δ2‖W1‖2

c2p

c1

.

Because maxi(1−wi) = o(n−1/2), ‖W1‖2 = o(1). Then δ2‖W1‖2 c2p

c1
= o(1). So

E‖(β −β0)
T s1−w

n1
(β0)‖2→ 0. This proves that the first term is op(1).

In order to show the second term is op(1), we need to show ‖(β−β0)
T∇s1−w

n1
(γ∗)(β−β0)‖

is op(1). Since

(β −β0)
T∇s1−w

n1
(γ)(β −β0) = δ

2λT I−1/2
n1
(β0)∇s1−w

n1
(γ)I−1/2

n1
(β0)λ,

it is sufficient to show

max
γ∈Nn1

(δ)
‖I−1/2

n1
(β0)∇s1−w

n1
(γ)I−1/2

n1
(β0)‖ → 0.

Let

Rw
n1
(β) =

n1∑

j=1

(1−w1 j)[y1 j −µ(ψ(β T X j))]ψ
′′(β T X j)t jX jX

T
j ,

M w
n1
(β) =

n1∑

j=1

(1−w1 j)[ψ
′(β T X j)]

2ζ′′(ψ(β T X j))t iX jX
T
j .

Then

∇s1−w
n1
(γ) = [Rw

n1
(γ)−M w

n1
(γ)]/φ.

So it suffices to show

max
γ∈Nn1

(δ)
‖M−1/2

n1
M w

n1
(γ)M−1/2

n1
‖ → 0,



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 348

and

max
γ∈Nn1

(δ)
‖M−1/2

n1
Rw

n1
(γ)M−1/2

n1
‖ → 0.

Because

‖M−1/2
n1

M w
n1
(γ)M−1/2

n1
‖ ≤max

j
(1−w1 j)‖M−1/2

n1
Mn1
(γ)M−1/2

n1
‖.

Using similar argument as in [1], ‖M−1/2
n1

Mn1
(γ)M−1/2

n1
‖ is bounded by

max
γ∈Nn1

(δ)
‖M−1/2

n1
Mn1
(γ)M−1/2

n1
‖ ≤ pp max

γ∈Nn1
(δ), j≤n1

|ϕ(γT X j)/ϕ(β
T X j)|,

which converges to 0 since ϕ is continuous and, for γ ∈ Nn1
(δ), |γT X j−β T X j|2→ 0. Because

max j(1−w1 j)→ 0, we have

max
γ∈Nn1

(δ)
‖M−1/2

n1
M w

n1
(γ)M−1/2

n1
‖ → 0.

Let

e j = y1 j −µ(ψ(βT X j)),

Uw
n1
(γ) =

n1∑

j=1

(1−w1 j)[µ(ψ(β
T X j))−µ(ψ(γT X j))]ψ

′′(β T X j)t jX jX
T
j ,

V w
n1
(γ) =

n1∑

j=1

(1−w1 j)e j[ψ
′′(γT X j)−ψ′′(β T X j)]t iX jX

T
j ,

W w
n1
(β) =

n1∑

j=1

(1−w1 j)e jψ
′′(β T X j)]t iX jX

T
j .

Then Rw
n1
(γ) = Uw

n1
(γ) + V w

n1
(γ) +W w

n1
(β). Using a similar argument as above, we can show

that

max
γ∈Nn1

(δ)
‖M−1/2

n1
Uw

n1
(γ)M−1/2

n1
‖ → 0.

Note that ‖M−1/2
n1

Uw
n1
(γ)M−1/2

n1
‖ is bounded by the product of

max
j
(1−w1 j)M

−1/2
n1

n1∑

j=1

|e j |t jX jX
T
j M−1/2

n1
= op(1)

and

max
γ∈Nn1

(δ), j≤n1

|ψ′′(γT X j)−ψ′′(β T X j)|,

which can be shown to be o(1) using the same argument. Hence,

max
γ∈Nn1

(δ)
‖M−1/2

n1
V w

n1
(γ)M−1/2

n1
‖ → 0.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 349

Finally we need to show

max
γ∈Nn1

(δ)
‖M−1/2

n1
W w

n1
(β)M−1/2

n1
‖ → 0.

Since E(e j) = 0 and e j ’s are independent, it suffices to show that

n1∑

j=1

E|(1−w1 j)e jψ
′′(β T X j)t iX

T
j M−1

n1
X j|1+τ→ 0

for some τ ∈ (0,1). It is trivial since max j(1−w1 j)→ 0 and it is shown in [1] that

n1∑

j=1

E|e jψ
′′(β T X j)t iX

T
j M−1

n1
X j|1+τ→ 0.

Now we have shown that the first term and the remainder of the Taylor expansion of B are

both op(1). So B = op(1).

In the following we show that C is dominated by A. By assumption (A5), we have

E|C | = E

������

n2∑

j=1

w2 j

�
l2 j(β)− l2 j(β0)

�
������

≤
n2∑

j=1

w2 j E

�����log
f (y2 j ,β)

f (y2 j ,β0)

�����

≤ K

n2∑

j=1

w2 j

= K · n2o(n−1/2)

→ 0.

This implies C →p 0.

Theorem 3. Under the assumptions of Theorem 2,

I1/2
n1
(β0)(β̂ − β0)→ Np(0, Ip)

in distribution.

Proof. Let

sw
n (β) =

∂

∂β

2∑

i=1

ni∑

j=1

wi j li j(β)

and ∇sw
n (β) =

∂

∂ β
sw
n (β). Then

∇sw
n (β) =∇sw

n1
(β) +∇sw

n2
(β),



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 350

where ∇sw
n1
(β) and ∇sw

n2
(β) are similarly defined on main group and contamination group,

∇sw
n1
(β) =

∂

∂β

n1∑

j=1

w1 j l1 j(β),

∇sw
n2
(β) =

∂

∂β

n2∑

j=1

w2 j l2 j(β).

Since C = op(1), we know that

∇sw
n2
(β)→ 0.

By the proof of B = op(1), we have

max
γ∈Nn1

(δ)
‖I−1/2

n1
(β0)∇s1−w

n1
(γ)I−1/2

n1
(β0)‖ → 0,

so, as ∇sn1
(γ) =∇s1−w

n1
(γ) +∇sw

n1
(γ) and Nn1

(δ) converges to β0,

I−1/2
n1
(β0)∇sw

n1
(β0)I

−1/2
n1
(β0)

= I−1/2
n1
(β0)∇sn1

(β)I−1/2
n1
(β0) + o(1)

→−Ip,

and

∇sw
n1
(β0)→−In1

(β0).

Then

∇sw
n (β0) =∇sw

n1
(β0) +∇sw

n2
(β0)→−In1

(β0).

Taking the Taylor expansion of sw
n (β̂) at β0, where β̂ is the WL estimate, it follows that

sw
n (β̂) = sw

n (β0) +∇sw
n (β0)(β̂ −β0) + op(∇sw

n (β0)(β̂ −β0))

and so

sw
n (β̂) = sw

n (β0)− In1
(β0)(β̂ −β0) + op(In1

(β0)(β̂ −β0)) (11)

Setting sw
n (β̂) = 0, and multiplying the both sides of (11) by I−1/2

n1
(β0), we obtain

I1/2
n1
(β0)(β̂ −β0) = I−1/2

n1
(β0)s

w
n (β0) + op(1).

Because C = op(1), we have var(sw
n2
(β))→ 0, then

var(sw
n (β0)) = var(sw

n1
(β0)) + var(sw

n2
(β0))

=
1

φ




n1∑

j=1

w2
1 jϕ((β

T
0 X j))t1 jX jX

T
j


+ op(1)



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 351

= In1
(β0)−

1

φ




n1∑

j=1

(1−w2
1 j)ϕ((β

T
0 X j))t1 jX jX

T
j


+ op(1)

= In1
(β0)− op(var(sw

n (β0))),

and hence

In1
(β0) = var(sw

n (β0)) + op(var(sw
n (β0))).

By the CLT [Corrollary 1.3, 1] and Slutsky’s theorem, we have

I−1/2
n1
(β0)s

w
n (β0)

d→ Np(0, Ip),

and hence

I1/2
n1
(β0)(β̂ −β0)

d→ Np(0, Ip).

4. Empirical Distribution by Bootstrap

We have given the asymptotic distribution of WLE, however, the asymptotic distribution is

hard to use in real data analysis. At the same time, the sample size of data sometime is not

big enough, and therefore it may cause bias to use the asymptotic distribution. In order to

assess the significance of a parameter based on WLE, we proposed the bootstrap method to

generate the empirical distribution of WLE.

Without loss of generality, we assume that m = 2 and number of independent variables

to be 1. Suppose the data we have is (x i, yi) for i = 1, . . . , n. Below are the main steps to

generate the empirical distribution.

Step 1: In the main group, let z j ( j = 1, . . . , k) to be the unique values of x i (i = 1, . . . , n1),

where k ≤ n1. Calculate the frequency of z j and let p0 =
∑

yi/n1.

Step 2: Draw n1 sample from z j , with probabilities corresponds to frequencies of zi. Denotes

the new sample of the main group as x ′i (i = 1, . . . , n1).

Step 3: Regenerate y ′i (i = 1, . . . , n1) from binomial distribution bin(n1, p0).

Step 4: Combine (x ′
i
, y ′

i
) (i = 1, . . . , n1) and (x i, yi) i = n1 + 1, . . . , n, calculate the WLE of

this new sample.

Repeat step 2 to 4 for N times, to get the empirical distribution of WLE with N values.

5. Simulation Study

In this section we present the simulation results to illustrate the numerical performance of

the proposed methods. We mainly compare the estimated values and the standard deviations

based on the MLE and WLE estimators and present the ratios of the MSE of the WLE to the

MSE of the MLE under different simulation scenarios.



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 352

Example 1. The data are generated from models (8) with β1 = 1 and β2 = 0.92. Let X1 and X2

are independent and identically distributed with N(0,1). Also, we set that σ1 = σ2 = 0.3. The

simulation study is conducted as follows. We consider three scenarios: (i) n1 < n2; (ii) n1 = n2;

and (iii) n1 > n2. The range of n1 and n2 is between 8 and 90. For each fixed sample size, 1000

independent data sets are generated. Table 1 summarizes the results. It can be seen that both the

MLE and WLE are close to the true values. The standard deviations of the WLE’s are consistently

smaller than those of the MLE. Furthermore, the ratios of the MSE of the WLE to the MSE of the

MLE are always smaller than 1. This implies that the WLE could reduce the MSE in contrast with

the MLE. To assess the sensitivity of the ratio to actual value of the related parameter β2, we run

simulations with different β2 values ranging from 0.7 to 0.98. The trend of the ratios against the

values of β2 is provided in Figure 1. In general, we find that the smaller n1, the smaller the ratio

under the current setup. For each fixed n1, the ratios are inversely proportional to the value of

β2. They would also increase with larger value for n2. We see that the WLE outperforms the MLE

remarkably in small sample size n1. This feature could be useful in practice if the first sample size

is small and we have abundant related information.

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=10;n2=8

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=10;n2=10

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=10;n2=15

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=20;n2=16

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=20;n2=20

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=20;n2=30

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=30;n2=24

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=30;n2=30

0.70 0.80 0.90

0.
6

0.
8

1.
0

1.
2

beta2

ra
tio

n1=30;n2=45

Figure 1: The trend of the ratio values against β2 for 9 
ombinations of (n1, n2). The horizontal line

indi
ates ratio= 1.



P.
G

u
o
,

X
.

W
a
n

g
,
Y.

W
u
/

E
u

r.
J.

P
u

re
A

p
p

l.
M

a
th

,
5

(2
0

1
2

),
3

3
3

-3
5

6
3

5
3

Table 1: Estimated values based on the MLE and WLE, the 
orresponding MSE and the ratio of the MSE(WLE) to the MSE(MLE) for Example

1.

MLE WLE

n1 n2 β̂(std) MSE(std) β̂(std) MSE(std) λ̂(std) Ratio

10 8 1.009(0.239) 0.057(0.093) 0.997(0.21) 0.044(0.069) 0.744(0.187) 0.768

20 16 1.008(0.166) 0.028(0.041) 0.992(0.149) 0.022(0.033) 0.757(0.186) 0.807

30 24 1.006(0.128) 0.016(0.025) 0.993(0.113) 0.013(0.02) 0.754(0.188) 0.778

40 32 1.001(0.114) 0.013(0.019) 0.991(0.102) 0.01(0.016) 0.767(0.187) 0.813

50 40 1.005(0.098) 0.01(0.014) 0.994(0.09) 0.008(0.012) 0.781(0.194) 0.851

60 48 0.999(0.094) 0.009(0.013) 0.99(0.088) 0.008(0.012) 0.774(0.192) 0.875

10 10 1.004(0.241) 0.058(0.085) 0.985(0.206) 0.042(0.063) 0.742(0.186) 0.734

20 20 1.011(0.164) 0.027(0.04) 0.996(0.143) 0.02(0.032) 0.743(0.188) 0.751

30 30 1.003(0.134) 0.018(0.026) 0.989(0.117) 0.014(0.02) 0.758(0.188) 0.765

40 40 1.001(0.112) 0.013(0.018) 0.988(0.1) 0.01(0.015) 0.761(0.188) 0.808

50 50 0.994(0.099) 0.01(0.015) 0.981(0.088) 0.008(0.012) 0.764(0.189) 0.812

60 60 0.999(0.088) 0.008(0.011) 0.986(0.081) 0.007(0.009) 0.77(0.187) 0.867

10 15 0.986(0.249) 0.062(0.108) 0.966(0.196) 0.04(0.077) 0.717(0.181) 0.638

20 30 1.006(0.161) 0.026(0.039) 0.98(0.131) 0.017(0.026) 0.72(0.186) 0.675

30 45 0.995(0.13) 0.017(0.025) 0.977(0.106) 0.012(0.018) 0.74(0.184) 0.693

40 60 1.002(0.109) 0.012(0.017) 0.984(0.095) 0.009(0.014) 0.753(0.188) 0.78

50 75 1.002(0.101) 0.01(0.015) 0.982(0.084) 0.007(0.011) 0.754(0.184) 0.729

60 90 1.002(0.096) 0.009(0.014) 0.986(0.086) 0.007(0.011) 0.761(0.185) 0.815



P. Guo, X. Wang, Y. Wu / Eur. J. Pure Appl. Math, 5 (2012), 333-356 354

6. A Real Data Example

The low birth weight data comes from a study of risk factors associated with low infant

birth weight.The data were collected at Baystate Medical Center, Springfield, Massachusetts,

during 1986 and presented in [8]. The goal of this study was to identify risk factors associated

with giving birth to a low birth weight baby who weighs less than 2500 grams. In this study

data were collected on 189 women, 59 of which had low birth weight babies and 130 of

which had normal birth weight babies. Variables which were thought to be of importance

were weight of the subject at her last menstrual period and race. These two variables are

denoted as "LWT" and "RACE". The response "LOW", is a binary variable which indicates a low

birth weight baby if equals to 1, and normal birth weight if equals to 0. The variable "RACE"

has been recoded using the two dummy variables. First of all, we investigate on the model

g(µ) = β0 + β1 LW T,

where µ is the expected value of the binary response and g is the link function for binomial

family. We find that the variable LWT is significant. The estimate of the coefficients and the

corresponding p-values are given in Table 2. One can also verify that the interaction between

LWT and RACE is not significant.

Table 2: Estimated Coe�
ients for general logisti
 regression model using the variables LWT.

Estimate p-value

Intercept 0.9983 0.2036

LWT -0.0141 0.0227

Then we check the significance of LWT within each RACE individually. The p-values are

all greater than the p-value of LWT for each group is given in Table 3. Despite the fact that

the variable LWT is significant for the overall model, it is very surprising to see that the LWT

is not significant within any group. By applying the WLE method based on distribution from

bootstraping, we find the LWT is significant in "Other" group.

Table 3: Estimated Coe�
ients for general logisti
 regression model using the variables LWT in ea
h RACE

group.

Estimate p-value

“White” Group

Intercept 0.7925 0.528

LWT -0.0151 0.123

“Black” Group

Intercept 0.6941 0.663

LWT -0.0069 0.517

“Other” Group

Intercept 2.7135 0.1066

LWT -0.0275 0.0559



REFERENCES 355

Table 4: Estimated Coe�
ients for weighted logisti
 regression model using the variables LWT with ea
h

RACE group as the main group. P-values are based on bootstrap distribution.

Estimate p-value

“White” Group

Intercept 0.7925 0.1296

LWT -0.0148 0.1332

“Black” Group

Intercept 0.6941 0.5396

LWT -0.0075 0.5788

“Other” Group

Intercept 2.7135 0.0408

LWT -0.0276 0.0432

7. Discussion

To obtain a more efficient estimator with a smaller MSE when related information is avail-

able, we proposed the weighted likelihood inference for the parameters in linear and par-

tially linear models. We developed a data-driven approach to estimate the weights to address

the vexing issued in weighted likelihood inference. The proposed estimators have the same

asymptotically normal distribution as the maximum likelihood estimators. The advantage of

the WLE over the classical MLE was illustrated by simulation studies. Results from these simu-

lation studies suggest that the proposed estimators have prospective promises. The simplicity

and effectiveness of the proposed adaptive weights are also appealing.

We remark that the real data set used in this paper actually came from a longitudinal

study. Thus one must recognize the within-subject and between-subject variations. Further

investigation of the inference based on the weighted likelihood for mixed-effect models is

required.

ACKNOWLEDGEMENTS The research was partially supported by the Natural Sciences and

Engineering Research Council of Canada.

References

[1] J Shao. Mathematical Statistics. Springer, New York, 2003.

[2] J Staniswalis. The kernel estimate of a regression function in a likelihood-based models.

J. Am. Statist. Assoc., 84:276–83, 1989.

[3] M Stone. Strong inconsistencies from uniform priors. J. Am. Statist. Assoc., 71:114–116,

1976.

[4] R Tibshirani and T Hastie. Local likelihood estimation. J. Am. Statist. Assoc., 82:559–67,

1987.



REFERENCES 356

[5] X Wang. Approximating bayesian inference by weighted likelihood. Can. J. Statist.,

34:279–298, 2006.

[6] X Wang and J Zidek. Derivation of mixture distribution and weighted likelihood as

minimizers of kl-divergence subject to constraints. Ann. Inst. Statist. Math., 57:687–

701, 2005.

[7] X Wang and J Zidek. Selecting likelihood weights by cross-validation. Ann. Statist.,

33:463–500, 2005.

[8] D. Hosmer and S. Lemeshow. Applied Logistic Regression. Wiley, New York, 1989.

[9] F. Hu. The asymptotic properties of the maximum-relevance weighted likelihood esti-

mators. Can. J. Statist., 25:45–59, 1997.

[10] F. Hu and J. Zidek. The relevance weighted likelihood with applications. In Ahmed, S.E.

& Reid, N. , editor, Empirical Bayes and Likelihood Inference., pages 211–235, New York.,

2001. Springer Verlag.

[11] F Hu and J Zidek. The weighted likelihood. Can. J. Statist., 25:347–71, 2002.

[12] F. Hu and W. Rosenberger. Analysis of time trends in adaptive designs with applications

to a neurophysiology experiments. Statist. Med., 19:2067–75, 2000.

[13] H. Akaike. Information theory and an extension of the maximum likelihood principle.

In Petrov, B.N. & Csaki, F. , editor, Proc. 2nd International Symposium on Information

Theory., pages 267–281, Budapest., 1973. Akademiai Kiado.

[14] I. Csiszár. I-divergence geometry of probability distributions and minimization prob-

lems. . Ann. Statist., 3:146–158, 1975.

[15] J. Bernardo. A maximum likelihood methodology for clusterwise linear regression. Ann.

Statist., 7:686–690, 1979.

[16] H Royden. Real Analysis. Prentice Hall, New York, 3 edition, 1988.

[17] S. Eguchi and J. Copas. A class of local likelihood methods and near-parametric asymp-

totics. J. R. Statist. Soc. B, 60:709–24, 1998.

[18] S Kullback. Information Theory and Statistics. Wiley, New York, 1959.

[19] T. Cover and J. Thomas. Elements of Information Theory. Wiley, New York, 1991.

[20] W James and C Stein. Estimation with quadratic loss. In Proc 4th Berkeley Symp. Math

Statist Prob., volume 1, pages 361–379. Berkely: University of California Press., 1961.

[21] X Wang, C van Emden and J Zidek. Asymptotic properties of maximum weighted likeli-

hood estimator. J. Statist. Plan. Infer., 119:37–54, 2004.