Ratio Mathematica Vol. 32, 2017, pp. 3–19 ISSN: 1592-7415 eISSN: 2282-8214 Some improved mixed regression estimators and their comparison when disturbance terms follow Multivariate t-distribution Manoj Kumar1, Vikas Bist2 and Man Inder Kumar3,∗. 1Department of Statistics, Panjab University, Chandigarh, India mantiwa@gmail.com 2Department of Mathematics, Panjab University, Chandigarh, India bistvikas@gmail.com 3Department of Statistics, Panjab University, Chandigarh, India maninderbajarh@gmail.com Received on: 27-05-2017. Accepted on: 15-06-2017. Published on: 30-06-2017 doi:10.23755/rm.v32i0.330 c©Manoj Kumar et al. Abstract The Mean square error matrices, bias vector and risk functions of pro- posed improved mixed regression estimators are obtained by employing the small disturbance approximation technique under the condition, when dis- turbance terms follows multivariate t-distribution. Further, the risk function criterion is used to examine the efficiency of proposed improved mixed re- gression estimators. Keywords: Stochastic restrictions; Mixed regression estimator; Stein- rule estimator; Multivariate t-distribution etc. 2010 AMS subject classifications: 62J05 3 Manoj Kumar, Vikas Bist and Man Inder Kumar 1 Introduction When incomplete prior information is expressible in the form of set of linear stochastic restrictions on the coefficients in a linear regression model, the method of mixed regression for the estimation of regression coefficients provides asymp- totically a more efficient estimator than the least squares method that ignores the prior restrictions. Stemming from the philosophy of stein-rule in this paper we proposed two fami- lies of improved estimators for the regression coefficients and study their proper- ties when disturbances have multivariate t-distribution. For multivariate t - distri- bution see, [12], [10] and [3]. In section 2, we discuss the framework and estima- tors. The properties of these estimators are presented in section 3 and the results are compared in section 4. Simulation Study is carried out to support theoretical finding in Section 5. 2 Model Specification and the Estimators Let us postulate the linear regression model Y = Xβ + U (1) Where, Y is a n × 1 vector of dependent variables; X is a n × p column rank matrix of n-observations on p explanatory non-stochastic variables; β is a p × 1 non-null vector of regression coefficient and U is a n × 1 vector of disturbance following multivariate student t-distribution with probability density function as: f ( U v ,σ2 ) = γv/2Γ ( v+n 2 ) π n 2 Γ ( v 2 ) σ−n [v + U ′U σ2 ]−n+v 2 (2) Where, v > 0,σ > 0 are respectively the degree of freedom and dispersion pa- rameters; the vector U has its error components Ui ∈ (−∞,∞), i = 1, 2, ...,n. Here the error vector U has mean vector E(U) = 0 for v > 1, variance-covariance matrix E(U ′U) = σ2 ( v v−2 ) I, for v > 2, measure of skewness γ1 = 0 and mea- sure of kurtosis γ2 = σ4 ( 6 v−4 ) I for v > 4. Let the stochastic restrictions on β in (1) be r = Rβ + V (3) Where, r is a J × 1 vector of known elements, R is a J ×p full row rank matrix of known elements and V is a J × 1 vector of distribution such that E(V ) = 0 ; E(V ′V ) = Ω (4) 4 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution Where, Ω is a J ×J positive definite symmetric matrix of known elements. Further, we assume that the errors associated with the stochastic restriction are independent with the distribution in model (1). The ordinary least square (OLS) estimator of β that ignores the prior restrictions (3) is bo = (X ′X)−1X′Y (5) If we consider the prior information (3), then the mixed regression (MR) estimator of β is given by bMR = [X ′X + S2R′Ω−1R]−1[X′Y + S2R′Ω−1r] (6) Where , S2 = 1 n−p ((Y −Xb)′(Y −Xb)) (7) The Stein-rule estimator of β is bs = [ 1 −k (Y −Xb)′(Y −Xb) b′o(X ′X)b0 ] bo (8) Where, k is a positive scalar characterizing the estimator. The Stein-Mixed Regression (SMR) estimator of β is given as bSMR = [ X′X + 1 n−p [(Y −XbS)′(Y −XbS)]R′Ω−1R ]−1 [ X′Y + 1 n−p [(Y −XbS)′(Y −XbS)]R′Ω−1r ] (9) The Mixed Stein-Regression (MSR) estimator of β is bMSR = [ 1 −k (Y −XbMR)′(Y −XbMR) b′MR(X ′X)bMR ] bMR (10) 3 Properties of the Estimators PX = X(X ′X)−1X′ (11) M = [I −PX ] (12) 5 Manoj Kumar, Vikas Bist and Man Inder Kumar Mj = [PX − jC−1Xββ′X′] j = 1, 2, . (13) Nj = [(X ′X)−1 − jC−1ββ′] j = 1, 2, . (14) C = β′X′Xβ (15) µ = (X′X)−1R′Ω−1R(X′X)−1 (16) The OLS estimator defined in (5) is found to be unbiased if v > 1, with variance - covariance matrix and risk function given by E[(b0 −β)(b0 −β)′] = σ2 ( v v − 2 ) (X′X)−1; v > 2 (17) Risk(bo) = σ 2 ( v v − 2 ) tr(X′X)−1L; v > 2 (18) Where, L is a positive definite symmetric loss matrix. The properties of the MR estimator are same as the SMR estimator, so we consider only the SMR estimator and present the results in the form of following theorems. Theorem 3.1. The asymptotic expression for the bias vector, mean squared error matrix and risk function of SMR estimator, up to order o(σ4) of approximations are given as B(bSMR) = 0 (19) M(bSMR) = σ 2 ( v v − 2 ) (X′X)−1 −σ4V1; v > 4 (20) Where, V1 = [( 1 − 2 n−p − 6 v − 4 θ ) µ + 6 (v − 4)(n−p)( µX′(In ∗M)X(X′X)−1 + (X′X)−1X′(In ∗M)Xµ )] (21) θ = trM(In ∗M) (n−p)2 (22) 6 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution Risk(bSMR) = σ 2 ( v v − 2 ) tr(X′X)−1L−σ4trV1L (23) Proof 3.1: To employ small disturbances asymptotic approximations. Let us write model (1) as Y = Xβ + σω (U = σω) (24) So that the i.i.d. elements of ω have multivariate-t distribution with mean zero for v > 1, variance ( v v−2 ) , for v > 2, measure of skewness γ1 = 0 and measure of kurtosis γ2 = ( 6 v−4 ) for v > 4. Now, using (24) in (5), we find b0 = β + σ(X ′X)−1X′ω (25) So that Y −Xb0 = σMω (26) Where M = [In −X(X′X)−1X′] (27) Using (25), we find up to order o(σ) of approximations. 1 b′o(X ′X)b0 = C−1[1 − 2σC−1β′D′ω] (28) Now, using (25), (26), and (28) in (8), we get up to order o(σ2) of approxima- tions. bs −β = σ(X′X)−1X′ω −σ2kω′MωC−1β (29) and for the same order of approximation, we have Y −Xbs = σMω −σ2kω′MωC−1Xβ (30) Thus, using (30) and (3) in (9), we get bSMR −β = σh1 + σ2h2 + σ3h3 + σ4h4 (31) Here, h1 = (X ′X)−1X′ω (32) h2 = ( ω′Mω n−p ) (X′X)−1R′Ω−1V (33) 7 Manoj Kumar, Vikas Bist and Man Inder Kumar h3 = ( ω′Mω T −G ) µX′ω (34) h4 = ( ω′Mω n−p )2 [k2(n−p)C−1(X′X)−1R′Ω−1V −µR′Ω−1V ] (35) It is easy to see that E(h1) = E(h2) = E(h3) = E(h4) = 0 (36) Utilizing (36) in (31), we obtain the result (19) of the Theorem 1. Now using (31), we get (bSMR −β)(bSMR −β)′ = σ2h1h′1 + σ 3(h1h ′ 2 + h2h ′ 1) + σ4(h1h ′ 3 + h2h ′ 2 + h3h ′ 1) (37) Here, E(h1h ′ 1) = (X ′X)−1 (38) E(h1h ′ 2) = E(h2h ′ 1) = 0 (39) E(h1h ′ 3) = 1 n−p [ 6 v − 4 (X′X)−1X′(In ∗M)Xµ + (n−p)µ ] (40) E(h2h ′ 2) = [( 6 v − 4 ) θ + ( n−p + 2 n−p )] µ (41) Utilizing (38), (39), (40) and (41) in (37), we obtain the result (20) of the Theorem 1. Risk(bSMR) = trM(bSMR)L (42) Thus, result (23) of the Theorem 1 follows from (42). Theorem 3.2. The asymptotic expression for bias vector, mean squared error ma- trix and risk function of MSR estimator, up to order o(σ4) of approximations are given as B(bMSR) = −σ2 kv(n−p) v − 2 C−1β + σ4 [ 6k v − 4 C−2( (trM4(In ∗M))I + 2(X′X)−1X′(In ∗M)X −Cθµ(X′X) ) β + kC−2 ( (n−p)(p− 2)I − n−p + 2 n−p Cµ(X′X) ) β ] (43) 8 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution Where * denotes Hadamard product. M(bMSR) = σ 2 ( v v − 2 ) (X′X)−1 −σ4 [ V1 + 12k v − 4 C−1[ (X′X)−1X′(In ∗M)X(X′X)−1 −C−1 ( (X′X)−1X′(In ∗M)Xββ′ + ββ′X′(In ∗M)X(X′X)−1 + ( k 2 ) (trM(In ∗M))ββ′ )] + 2k(n−p)N(2+ k 2 (n−p+2)) ] (44) Risk(bMSR) = σ 2 ( v v − 2 ) tr(X′X)−1L−σ4 [ trV1L + 12 k v − 4 C−1 ( tr(X′X)−1X′(In ∗M)X(X′X)−1L −C−1 ( 2β′X′(In ∗M)X(X′X)−1Lβ + k 2 (trM(In ∗M))β′Lβ )) + 2k(n−p)trN(2+ k 2 (n−p+2))L ] (45) Proof 3.2: Using (3), (24) and (26) in (6), we obtain up to order o(σ2) of approximations. bMR = β + σ(X ′X)−1X′ω + σ2 ( ω′Mω n−p ) (X′X)−1R′Ω−1V (46) Thus, for the same order of approximation, we have 1 b′MR(X ′X)bMR = C−1 [ 1 − 2σC−1β′X′ω −σ2C−1 ( 2 n−p ω′MωV ′Ω−1Rβ + ω′MDω )] (47) Using (46), we get up to order o(σ2)of approximations. Y −XbMR = σMω −σ2 ( ω′Mω n−p ) X(X′X)−1R′Ω−1V (48) Using (46), (47) and (48) in (10), we obtain up to order o(σ4), we get bMSR −β = σh∗1 + σ 2h∗2 + σ 3h∗3 + σ 4h∗4 (49) Where h∗1 = (X ′X)−1X′ω (50) 9 Manoj Kumar, Vikas Bist and Man Inder Kumar h∗2 = ( ω′Mω n−p )[ (X′X)−1R′Ω−1V −kC−1β ] (51) h∗3 = − ( ω′Mω n−p )[ µ + k(n−p)C−1N2) ] X′ω (52) h∗4 = k(ω ′Mω)C−2 ( 2 n−p ω′Mωββ′R′Ω−1V + ω′M4ωβ + 2(X′X)−1X′ωω′Xβ ) − ( ω′Mω n−p )2 [ µR′Ω−1V + kC−1(β′V ′Ω−1R + (n−p)I)(X′X)−1R′Ω−1V ] (53) Here, it is easy to verify that E(h∗1) = 0 (54) E(h∗2) = −k(n−p)C −1β (55) E(h∗3) = 0 (56) E(h∗4) = 6k v − 4 C−2 [ (trM4(In ∗M))I + 2(X′X)−1X′(In ∗M)X −Cθµ(X′X) ] β + kC−2 [ (n−p)(p− 2)I − ( n−p + 2 n−p ) Cµ(X′X) ] β (57) Utilizing (54), (55), (56) and (57) in (53), we obtain the result (43) of the Theorem 2. Now, using (53) we get (bMSR −β)(bMSR −β)′ = σ2h∗1h ∗′ 1 + σ 3(h∗1h ∗′ 2 + h ∗ 2h ∗′ 1 ) + σ4(h∗1h ∗′ 1 + h ∗ 2h ∗′ 2 + h ∗ 3h ∗′ 1 ) (58) Here, we see that E(h∗1h ∗′ 1 ) = ( v v − 2 ) (X′X)−1 (59) E(h∗1h ∗′ 2 ) = 0 (60) E(h∗1h ∗′ 3 ) = 6 (v − 4)(n−p) [ (X′X)−1X′(In ∗M)Xµ + k(n−p)C−1(X′X)−1X′(In ∗M)XN2 ] −µ−k(n−p)C−1N2 (61) 10 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution E(h∗2h ∗′ 2 ) = µ [ 6 v − 4 θ + ( n−p + 2 n−p ) I ] + k2C−2ββ′ [ 6 v − 4 trM(In ∗M) + (n−p)(n−p + 2) ] (62) Utilizing (59), (60), (61) and (62) in (58), we obtain the result (44) of the Theorem 2. Similarly, we can obtain the result (45) of the Theorem 2. 4 Comparison of the Estimators 4.1 The comparison the risk functions of OLS and SMR esti- mators On comparison the risk functions of OLS and SMR estimators. We observe that up to order o(σ2) of approximations, both the estimators have same risk and for higher order of approximation, we see that Risk(b0) −Risk(bSMR) = σ4 [ 6 v − 2 ( 2 n−p tr(X′X)−1X′(In ∗M)XµL−θtrµL ) + (n−p− 2 n−p ) trµL ] (63) If we choose L = (X′X), then expression (63) becomes Risk(b0) −Risk(bSMR) = σ4 [ 6 v − 2 ( 2 n−p tr(X′X)−1X′(In ∗M)X(X′X)−1R′Ω−1R −θtr(X′X)−1R′Ω−1R ) + ( n−p− 2 n−p ) tr(X′X)−1R′Ω−1R ] (64) Since, the expression (64) is positive semi-definite, so bSMR dominates b0 and as v →∞, expression (64) reduces to Risk(b0) −Risk(bSMR) = σ4 ( n−p− 2 n−p ) tr(X′X)−1R′Ω−1R (65) Which is positive semi-definite. Thus, bSMR dominates b0, so long as n−p > 2. 11 Manoj Kumar, Vikas Bist and Man Inder Kumar 4.2 The comparison the risk functions of OLS and MSR esti- mators On comparison the risks of OLS and MSR, we see that up to order o(σ2) of approximations, both the estimators have same risk and for higher order of approximations, we find that bMSR dominates bo so long as (65) is positive semi- definite and if we choose k to satisfy, 0 < k < 2(n−p) T C β′Aβ [ tr(X′X)−1L− 2C−1β′Lβ + 6 (n−p)(v − 4)( tr(X′X)−1X′(In ∗M)X(X′X)−1L− 2C−1β′X′(In ∗M)X(X′X)−1Lβ )] (66) Where T = [ 6 v − 4 (trM(In ∗M)) + (n−p)(n−p + 2) ] (67) If we choose L = (X′X), then the above condition of dominance becomes 0 < k < 2(n−p) T [ p− 2 + 6 (n−p)(v − 4) ( tr(X′X)−1X′(In ∗M)X − 2C−1β′X′(In ∗M)Xβ )] (68) And as v →∞, condition (68) reduces to 0 < k < 2 n−p + 2 (p− 2); p > 2 (69) Which is well known condition of dominance of stein-rule estimator over the least squares estimator. 4.3 The comparison the risk functions of SMR and MSR esti- mators On comparing the risk function associated with the estimators SMR and MSR respectively, we observe that the estimator MSR dominates the estimator SMR so long as (30 )holds and as v → ∞ and again by choosing L = (X′X), the condition of dominance becomes (69). 12 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution 5 Simulation Results The proposed estimator bSMR is more efficient than OLSE under given lin- ear model. Although, theoretically the results are drawn in equation (65), the proposed Stein-mixed Regression (SMR) estimator bSMR is more efficient than ordinary least square estimator b0 under condition n−p > 2. In this section, we perform simulations for exact equation (65) under conditions n−p > 2,n > p > j with sigma equal to one. Each result is based on 100,000 simulations runs using MATLAB. The result shown for n = 10, 11, 12, 13, 14, 15 in Table 1, 2, 3 & 4. The main finding of our numerical evaluation is following:- 1. The simulation results strongly support the theoretical findings. 2. The simulation result also explains the strength keep on increasing as we go for large value of n,p and j. 3. The results are independent of value of sigma. 4. Hence, bSMR is more efficient than b0 under condition n−p > 2. 5. The simulation results also reveals that bMSR is also more efficient over b0 (as it also depends on (65) under condition at (69)). Figure 1: Average dominance condition for difference between n & p Based on simulation study, the dominance of bSMR has been proven over b0 under certain set of conditions. Further, the behavior of dominance is studied for various combination of different values of n,p and j. The average dominance is derived based on probability for different combination of n,p and j; when σ = 1. The figure 1 depicts average dominance keeps on decreasing with increase in gap 13 Manoj Kumar, Vikas Bist and Man Inder Kumar Figure 2: Dominance behavior for different values of p; when n=10 & j=5 Figure 3: Average dominance condition for given value of p & j for n=20 between n and p. The figure 2 also depicts a decreasing trend with increase in value of p, when n = 10 and j = 5. Similarly, figure 3 shows the behavior of dominance condition for different value of p and j for fixed value of n equal to 20. 14 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution Table 1: Average Value of Dominance for different values of n and p for j = 2 sigma = 1. j=2 n=10 n=11 n=12 n=13 n=14 n=15 p=3 0.67823 0.67492 0.67793 0.67688 0.67390 0.67539 p=4 0.67300 0.67079 0.66877 0.66903 0.66816 0.66654 p=5 0.67187 0.66562 0.66558 0.66668 0.66416 0.66691 p=6 0.66660 0.66642 0.66755 0.66509 0.66370 0.66186 p=7 0.66755 0.66497 0.66408 0.65994 0.66305 0.65982 p=8 - 0.66816 0.66611 0.66010 0.66261 0.66156 p=9 - - 0.66861 0.66352 0.66143 0.66036 Remark: No value for dominance where n−p 6 2. 15 Manoj Kumar, Vikas Bist and Man Inder Kumar Table 2: Average Value of Dominance for different values of n and p for j = 3 sigma = 1. j=3 n=10 n=11 n=12 n=13 n=14 n=15 p= 4 0.70809 0.70806 0.70722 0.70725 0.70806 0.70611 p= 5 0.70570 0.70449 0.70581 0.70385 0.70381 0.70162 p= 6 0.70612 0.70319 0.70252 0.69998 0.70281 0.70205 p= 7 0.70651 0.70266 0.70281 0.70253 0.69810 0.69816 p= 8 - 0.70735 0.70167 0.70011 0.70183 0.69960 p= 9 - - 0.70592 0.70486 0.70001 0.70028 p=10 - - - 0.70784 0.70426 0.70033 p=11 - - - - 0.70692 0.70500 Remark: No value for dominance where n−p 6 2. 16 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution Table 3: Average Value of Dominance for different values of n and p for j = 4 sigma = 1. j=4 n=10 n=11 n=12 n=13 n=14 n=15 p= 5 0.74738 0.74891 0.74713 0.74538 0.74603 0.74668 p= 6 0.74694 0.74586 0.74771 0.74337 0.74335 0.74346 p= 7 0.75060 0.74841 0.74539 0.74412 0.74366 0.74381 p= 8 - 0.74740 0.74561 0.74405 0.74213 0.74607 p= 9 - - 0.74752 0.74623 0.74547 0.74207 p=10 - - - 0.74601 0.74774 0.74091 p=11 - - - - 0.74832 0.74426 p=12 - - - - - 0.74612 Remark: No value for dominance where n−p 6 2. 17 Manoj Kumar, Vikas Bist and Man Inder Kumar Table 4: Average Value of Dominance for different values of n and p for j = 5 sigma = 1. j=5 n=20 n=25 n=30 n=35 n=40 n=45 n=50 p = 5 0.78748 0.78793 0.78713 0.78373 0.78562 0.78336 0.78686 p=10 0.78359 0.78228 0.78284 0.78158 0.77898 0.77644 0.78005 p=15 0.78586 0.78143 0.77689 0.78072 0.77643 0.77454 0.77655 p=20 - 0.78350 0.78017 0.77976 0.77657 0.77576 0.77628 p=25 - - 0.78453 0.78062 0.77884 0.77852 0.77563 p=30 - - - 0.78408 0.78044 0.77743 0.77703 p=35 - - - - 0.78478 0.77772 0.77486 p=40 - - - - - 0.78208 0.77728 Remark: No value for dominance where n−p 6 2. References [1] Chaturvedi, A. and Shukla, G., Stein-rule estimation in linear models with non-scalar error covariance matrix, Sankhya, Series B, 52, (1990), 293-304. [2] Chaturvedi, A. ,Wan, A. T. K. and Singh, S. P., Stein-rule restricted regres- 18 Some improved MR estimators & their Comparison when disturbance terms follow Multivariate t-distribution sion estimator in a linear regression model with non-spherical disturbances, Communications in Statistics, Theory and Methods, 30, (2001), 55-68. [3] Giles, A.J., Pretesting for linear restriction in a regression model with spher- ically symmetric distributions, Journal of Econometrics, 50, (1991), 377- 398. [4] Judge, G. G. and Bock, M. E., The Statistical Implications of Pre-Test and Stein-Rule Estimators in Econometrics, North Holland, Amsterdam, 1978. [5] Kadane, J.B.,Comparison of k-class Estimators When disturbance are small, Econometrica, 39, (1971), 723-737. [6] Ohtani, K. and Wan, A. T. K., On the sampling performance of an improved Stein inequality restricted estimator, Australian and New Zealand Journal of Statistics, 40, (1998), 181-187. [7] Rao, C. R., Linear Statistical Inference and Its Applications, 2nd Edition. John Wiley,New York, 1973 . [8] Shalabh and Wan, A. T. K., Stein-rule estimation in mixed regression models, Biometrical Journal, 42, (2000) 203-214. [9] Sutradhar, B.C. and Ali, M.M., Estimation of parameters of regression with a Multivariate t-error variable, Communication Statistics - Theory and Meth- ods, A 15, (1986), 429-450. [10] Sutradhar, B.C., Testing Linear Hypothesis with t - Error Variable, Sankhya: The Indian Journal of Statistics, Series B (1960-2002), 50(2), (1988), 175- 180. [11] Theil, H., Principles of Econometrics, Vol. 1. Wiley, New York, 1971. [12] Zellner, A., Bayesian and non-Bayesian analysis of regression model with multivariate t-error terms, Journal of the American Statistical Association, 71, (1976), 400-405. 19