/compile/output.dvi EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS Vol. 8, No. 4, 2015, 431-449 ISSN 1307-5543 – www.ejpam.com On the Variance of Antithetic Time Series Dennis Ridley1,∗, Pierre Ngnepieba2 1 SBI, Florida A&M University and Department of Scientific Computing, Florida State University, Tallahassee, FL., USA. 2 Department of Mathematics, Florida A&M University, Tallahassee, FL, 32307, USA. Abstract. Combining antithetic time series is used to reduce model fitted and forecast mean square error (MSE). This is accomplished by removing the component of error that represents bias. The po- tential to reduce error is a function of variance in the time series. The greater the variance the greater is the potential for error reduction. But, the greater the variance the less is the efficacy of reversing corre- lation and combining antithetic time series. The percentage reduction in MSE increases with variance up to a limit then reduces. 2010 Mathematics Subject Classifications: AMS 62E20, 62F10 Key Words and Phrases: Antithetic time series theory, Time Series analysis, Bias reduction, Inverse correlation, Serial correlation, Forecasting 1. Introduction Combining antithetic time series to reduce fitted MSE has applications in statistical process control, and the analysis of engineering, scientific, medical and economic time series. The method was demonstrated by Ridley and Ngnepieba [30] with the well-known CompanyX data. They also showed that it satisfies the Diebold and Mariano [12] test for significance in forecast improvement. Before that, the problem of bias in CompanyX data ARIMA models was observed by Chatfield and Prothero [6]. The problem of bias in time series modeling in general was discussed by Copas [10], Griliches [14], Kendall [18], Klein [20], Koyck [21], Marriott and Pope [25], and Nerlov [26]. Prior to time series applications, complementary antithetic random numbers r and 1− r were suggested by Hammersley and Morton [16] for Monte Carlo computer simulation (see also Hammersley and Handscomb [15]). The results were mixed (see Kleijnen [19], Hendry [17], Ripley [32] and Davidson and McKinnon [11]). Clements and Hendry [9] discussed the use of antithetic variates to establish various properties concerning bias in forecasting models (see also Calzolari [5], Fisher and Salmon [13] and Mariano and Brown [24]). ∗Corresponding author. Email addresses: dridley@fsu.edu (D. Ridley), Pierre.Ngnepieba@famu.edu (P. Ngnepieba) http://www.ejpam.com 431 c© 2015 EJPAM All rights reserved. D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 432 Antithetic time series analysis as discussed in this paper is different. And, combining anti- thetic time series is fundamentally different from traditional combining. Traditional combining applies to independent estimates obtained from two or more different models, which may col- lectively contain more information than any one model. But, there is no explicit correction of systematic bias that might occur in any one or more of the models. For an extensive review of traditional combining see Armstrong [1], Bates and Granger [2], Bunn [4], Clemen [7], Clemen and Winkler [8], Makridakis and Hibon [23], and Makridakis, et al., [22]. Antithetic combining applies to two estimates. The first estimate is obtained from the single best model, subject only to possible systematic bias. The second estimate is created so as to be inversely correlated with the first, and thereby eliminate systematic bias (Ridley, [27–29]). Since Ridley [29] illustrated antithetic time series combining for an extensive range of autoregressive pro- cesses and probability distributions, those results will not be repeated here. Ridley, Ngnepieba and Duke [31] studied the effect of combining parameters on combining. In this paper we extend the simulation studies and derive a closed form formulae for MSE before and after combining. Like in the demonstration with actual data by Ridley and Ngnepieba [30], the variance of a real time series is fixed and cannot be altered. Therefore, this paper focuses on simulated lognormal AR(1) time series where the variance can be altered to illustrate its effect. The paper is organized as follows. The theory of correlation reversal and bias reduction as a function of variance is reviewed in sections 2 and 3. An analytical function for the percentage reduction in MSE is derived in section 4. The effect of variance on percentage reduction in MSE is illustrated by way of computer simulation in section 5. The conclusions in section 6 contain recommendations for future research. 2. Reverse Correlation and Variance Definition 1. Two random variables are antithetic if their correlation is negative. A bivariate collection of random variables is asymptotically antithetic if its limiting correlation approaches minus one asymptotically (see Ridley, [29] for a derivation). Definition 2. {X (ξ, t)} is an ensemble of random variables, where ξ belongs to a sample space and t belongs to an index set representing time, such that X t is a discrete realization of a lognormal stationary stochastic process from the ensemble, ln X t ∼ N(µ,σ), and X t , t = 1,2,3, . . . are serially correlated. Remark 1. Antithetic time series involves the application of a power transformation that cannot be applied directly to normally distributed numbers that by definition include negative numbers. The Ridley [29] antithetic time series theorem states that "if X t > 0, t = 1,2,3, . . . is a discrete realization of a lognormal stochastic process, such that ln X t ∼ N(µ,σ), then if the correlation between X t and X p t is ρX X p , then limp→0−,σ→0ρX X p = −1." The standard deviation σ and variance σ2 are for the logarithm of the time series. Therefore, in practice, they are naturally small. However, since there is no guarantee that the variance will actually approach zero, there will be some loss of efficacy in reversing correlation and combining antithetic time series. The purpose of this extension of Ridley and Ngnepieba [30] (and Ridley, [29] lognormal theory) is to illustrate the effect on percentage reduction in MSE for various values of σ2. D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 433 3. Antithetic Combining and Bias Reduction Before antithetic combining can correct bias in a time series model, bias must first occur. Bias occurs when a sample of data is observed, an autoregressive model is fitted to the data, and there is failure to realize all the assumptions of the model, leading to correlation between the error and the lagged variable. The antithetic combining model (Ridley, [29], Ridley and Ngnepieba, [30]), applied to fitted data, is summarized as follows: bX c,t = wbX t + (1−w) bX ′t , t = 1,2,3, . . . , n where bX t are fitted values obtained from a time series model X t = ΦX t−1+εt , t = 1,2,3, . . . n. The parameter−∞ < w<∞ is a combining weight. The fitted values bX t and bX ′t are antithetic in the sense that they contain components of error ε̂t and ε̂′t , respectively, that are biased and when weighted, wε̂t and (1 − w)ε̂′t are perfectly negatively correlated. The antithetic component bX ′t is estimated from bX ′t = X + rX̂ X̂ p � sX̂/sX̂ p ��bX p t − bX p � , t = 1,2, . . . , n where the exponent of the power transformation is set to the small negative value p = −0.001, r denotes sample correlation coefficient and s denotes sample standard deviation. In data analysis, data are often assumed to be normally distributed in order to use well known normal theory. But, the distribution of many data, including particle physics, engineer- ing, business and economic data is lognormal. For example, it is not possible to manufacture or sell a negative quantity, but the positive amount of the quantity is unlimited. Hence the distribution is positively skewed. In any case, if the data really are normally distributed, they can easily be transformed to a lognormal distribution by simple exponentiation. When the distribution of X t is lognormal, the distribution of limp→0− X p t is normal (Ridley, [29]). The exponent p is set to a small negative number to simulate the limit as p→ 0−. Since it is fixed, it is not an added model parameter that must be estimated from data. Also, the role of p is perfect reversal of correlation, not the conversion of a non-linear model to a linear model. The complete combining function for empirical data is given by bX c,t = wbX t + (1−w) n X + � 1− k p n− t + 1 � rẐ Ẑ p � sẐ/sẐ p ��bZ p t − bZ p �o , t = 1,2, . . . , n, where a shift parameter λ (suggested by Box and Cox, [3]) is used to facilitate the power transformation by adding λ to each X t to obtain Zt = X t + λ prior to applying the power transformation and subtracting it after conversion back to their original units, leaving the mean unchanged. Ridley [29] proved the counterintuitive result that rẐ Ẑ p is independent of λ. The values of k, λ and w are selected so as to minimize the combined fitted MSE in bX c,t . The expectation is that if bX t are biased, then bX c,t will exhibit diminishing bias as p approaches zero from the left (p inside an infinitesimal neighborhood of zero but not zero) and the correlation rẐ Ẑ p approaches −1. If bX t are unbiased then w= 1 and the combined fitted values are just the D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 434 original fitted values. The factor k p n− t + 1 is to correct for any apparent heteroscedasticity in the sample data, and k is expected to be small, or zero if there is none. The antithetic combining calculations are performed by FOURCAST†. 4. Analytic Per Unit Reduction in MSE Function of Variance In order to gain some insight into how antithetic combining is affected by variance, con- sider the stationary model X t = µX +Φ � X t−1 −µX � + ǫt , t = 2,3, . . . , n, (1) where X t are lognormally distributed, ln X t ∼ N(µ,σ), and whose fitted values are bX t = bµX + bΦ � X t−1 − bµX � , t = 2,3, . . . , n, (2) where bµX and bΦ are least-squares estimates of µX and Φ, respectively. The least squares esti- mate bΦ of Φ (see Ridley [29]) is bΦ = Φ+Cov � ǫt , X t−1 � /Var � X t � , (3) where fitted values bX t obtained from bµX +bΦ � X t−1 − bµX � are biased if Cov � ǫt , X t−1 � 6= 0. From (2) and (3), and given that the time series is stationary so that Var � X t � = Var � X t−1 � , then as n→∞ and bµX → µX , bX t =µX +Φ � X t−1 −µX � + � Cov � ǫt , X t−1 � /Var � X t � � X t−1 −µX � (4) =E � X t � + vt + ut , where vt are purely random errors and ut = � Cov � ǫt , X t−1 � /Var � X t � � X t−1 −µX � are sys- tematic serially correlated errors. From Appendix A, the total MSE before combining is given by (A1): MSE � σ2 � =  1− ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � From Appendix B, the MSE after combining is given by (B6): MSEc � σ2 � =  �w2 − 2w � ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1− 2 (1−w)exp � lnbX + s2/2 � σ2/ p Var(X t)p exp(σ2)− 1 + (1−w)2exp � 2lnbX + s2 � Var(ln bX t) Var(X t) †Application program EMC, Inc., http://www.fourcast.net/fourcast, Version 2012.1, File: CompanyX.zip D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 435 + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var(X t) « exp � lnbX + s2/2 � Cov � X t−1, ln bX t � Var � X t � � Var � X t � From Appendix C, per unit reduction in MSE due to combining is given by (C1): PMSEc � σ2 � = A+ B 1− § Φ+ Cov(ǫt ,X t−1) Var(X t) ª2 , where A=−   ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−1 − 1   2¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 +  1− ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−2   × σ2 � exp (σ2)− 1 � and B =− 2 ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−1  1− ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−1   ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × Cov � X t−1, ln bX t � exp ¦ µ+ σ 2 2 ©� exp (σ2)− 1 � and substituting for Var � X t � PMSEc � σ2 � = C + D 1− § Φ+ Cov(ǫt ,X t−1) exp{2µ+σ2}(exp(σ2)−1) ª2 , where C =−   ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−1 − 1   2¨ Φ+ Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «2 +  1− ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−2   σ2 � exp (σ2)− 1 � D =− 2 ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−1  1− ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−1   D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 436 × ¨ Φ+ Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � « Cov � ǫt , X t−1 � exp ¦ µ+ σ 2 2 ©� exp (σ2)− 1 � where Cov(ǫt ,X t−1) exp{2µ+σ2}(exp(σ2)−1) 6= 1. We are interested in the per unit reduction in MSE for time series with different variances. Setting µ= 0, PMSEc � σ2 � = E + F 1− § Φ+ Cov(ǫt ,X t−1) exp{σ2}(exp(σ2)−1) ª2 , (5) where E =−   ¨ 1− Cov � ǫt , X t−1 � exp {σ2} �exp (σ2)− 1 � «−1 − 1   2¨ Φ+ Cov � ǫt , X t−1 � exp {σ2} �exp (σ2)− 1 � «2 +  1− ¨ 1− Cov � ǫt , X t−1 � exp {σ2} �exp (σ2)− 1 � «−2   σ2 � exp (σ2)− 1 � , F =− 2 ¨ 1− Cov � ǫt , X t−1 � exp {σ2} �exp (σ2)− 1 � «−1  1− ¨ 1− Cov � ǫt , X t−1 � exp {σ2} �exp (σ2)− 1 � «−1   × ¨ Φ+ Cov � ǫt , X t−1 � exp {σ2} �exp (σ2)− 1 � « Cov � X t−1, ln X̂ t � exp ¦ σ2 2 ©� exp (σ2)− 1 � , and Cov(ǫt ,X t−1) exp{σ2}(exp(σ2)−1) 6= 1. This analytic function (5) for the per unit reduction in MSE due to combining is somewhat complicated. The errors ǫt are unobservable. Furthermore, since the model estimate of pa- rameter Φ is biased, its true value is unknown. The objective of antithetic time series analysis is not to correct the bias in the model parameter. The objective is to correct the bias in the model fitted values. Still, due to these unknowns, it is not possible to evaluate this function. An illustration of how combining reduced MSE, and how it changes with variance, can be accomplished by computer simulations. The simulations and results are explained below in Section 5. 5. Simulated Percentage Reduction in MSE and Variance The ability to reverse the correlation (limp→0−,σ→0ρX X p = −1) depends on small variance. Therefore, the ability to reduce bias and combined fitted MSE also depends on small variance. D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 437 However, the sample variance will depend on the particular data. To illustrate the effect of variance, let us consider the results of hypothetical simulations for the lognormal time series X t = exp � Yt � , where Yt = βYt−1 + et , t = 2,3, . . . , 1000, et ∼ N � 0,η � and σ2 = η2 (1−β2) . Let the model to be fitted to X t be X t = ΦX t−1+ εt , t = 2,3, . . . , 1000. The least squares estimate bΦ of Φ and the fitted values X̂ t obtained from bΦX t−1 will be biased if there is any deficiency (eg. inter alia, lack of normality, non-stationarity, unavoidably missing variables, sampling bias, serial correlation in the errors) resulting in the covariance Cov � ǫt , X t−1 � 6= 0. Since X t are positive and stationary, k and λ (see section 3) are both zero. Consider the case of η2 = 0, σ2 = 0. That is, no variance, no variations in the time series to be explained by the time series model, and no MSE. Then, no improvement in MSE is possible by antithetic combining. Antithetic combining does not include σ2 = 0. To simulate small variance, consider η2 = 0.01, σ2 = 0.027 and p = −0.001 as shown in Figure 1a. The correla- tion rX̂ X̂ p starts out at approximately −1 where the combining contribution to MSE reduction is near maximum. As the variance is increased, the amount of variance unexplained by the fitted model increases, contributing to the MSE improvement that is possible. At the same time, the correlation moves away from −1 (see Figure 1b), reducing the ability for antithetic combining to improve MSE. The net effect is illustrated below in Figures 2 and 3. (a) Correlation coefficient vs p (b) Correlation coefficient vs σ2 Figure 1: Correlation coefficient of simulated population values ρX X p and fitted values rX̂ X̂ p . Figure 2a shows the percentage improvement in MSE for different levels of variance (σ2 = 2,4,8,10) as the autoregressive coefficient β increases. For σ2 = 2, the percentage improvement increases until about β = 0.55 then falls. For larger σ2, the percentage improve- ment increases until about β = 0.8 then falls. Figure 2b shows the percentage improvement in MSE for different levels of autoregressive coefficient (β = 0.4,0.6,0.8) as σ2 increases. For all values of β , the improvement increases up to a maximum value then falls. For example, when β = 0.8, the maximum improvement in MSE is about 10.5% and occurs when σ2 is about 15. Therefore, in practice, antithetic combining will perform more or less well depend- ing on the variance of an actual time series. Forecasting is outside the scope of this paper, but Ridley and Ngnepieba [30] demonstrated how small idiopathic error bias accumulates to produce large ex ante multiple-period-ahead forecast errors that are corrected dynamically by antithetic combining. D. Ridley, P. Ngnepieba / Eur. J. Pure Appl. Math, 8 (2015), 431-449 438 (a) Percentage improvement in MSE vs β (b) Percentage improvement in MSE vs σ2 Figure 2: Percentage improvement in MSE for p = −0.001 and versus β and σ2. Figure 3a shows the percentage improvement in MSE for different levels of variance (η2 = 2,4,8,10) as the autoregressive coefficient β increases. For all levels of η2, the percent- age improvement increases until about β = 0.8 then falls. Figure 3b shows the percentage improvement in MSE for different levels of autoregressive coefficient (β = 0.4,0.6,0.8) as η2 increases. For all values of β , the improvement increases up to a maximum value then falls. For example, when β = 0.8, the maximum improvement in MSE is about 10.6% and occurs when η2 is about 4. (a) Percentage improvement in MSE vs β (b) Percentage improvement in MSE vs η2 Figure 3: Percentage improvement in MSE for p = −0.001 and versus β and η2. 6. Conclusions The variance of the logarithm of a time series is naturally small. Therefore, combining antithetic time series to reduce idiopathic bias in the fitted values from a time series model is generally applicable. Furthermore, as the variance increases, the percentage improvement in fitted MSE is actually better. This is due to the higher potential to improve MSE. As the variance continues to increase, thereby departing from the strict requirements of antithetic time series theory, the ability of antithetic combining to reverse correlation and improve MSE diminishes (some similarity in the loss of efficacy of feedforward control was reported by Shi and Kapur [34] [33]). As the two effects interact, the net percentage improvement in fitted MSE increases to a maximum then it declines. REFERENCES 439 ACKNOWLEDGEMENTS The authors would like to thank Minjing Tao for her helpful com- ments and suggestions to improve the clarity of the paper, and Dennis Duke for good discus- sions. References [1] J. S. Armstrong. Combining forecasts: The end of the beginning or the beginning of the end. International Journal of Forecasting, 5(4):585–588, 1989. [2] J. M. Bates and C. W. J. Granger. The Combination of Forecasts. Operational Research Quarterly, 20(4):451–468, 1969. [3] G. E. Box and D. R. Cox. An analysis of transformations. Journal of the Royal Statistical Society B, 26(2):211–252, 1964. [4] D. W. Bunn. The synthesis of predictive models in marketing research. Journal of Mar- keting Research, 16(May):280–283, 1979. [5] G. Calzolari. Antithetic variates to estimate the simulation bias in non-linear models. Economics Letters, 4(4):323–328, 1979. [6] C. Chatfield and D. L. Prothero. Box-Jenkins seasonal forecasting: problems in a case study. Journal of the Royal Statistical Society: Series A, 136(3):295–336, 1973. [7] R. T. Clemen. Combining forecasts: a review and annotated bibliography. International Journal of Forecasting, 5(4):559–583, 1989. [8] R. T. Clemen and R. L. Winkler. Combining economic forecasts. Journal of Business and Economic Statistics, 4(1):39–46, 1986. [9] M. P. Clements and D. F. Hendry. Forecasting Economic Time Series. University Press, Cambridge: Cambridge, 1998. [10] J. B. Copas. Monte Carlo results for estimation in a stable Markov time series. Journal of the Royal Statistical Society, Series A, 129(1):110–116, 1966. [11] R. Davidson and McKinnon. Estimation and Inference in Econometrics. Oxford University Press, Oxford, 1993. [12] F. X. Diebold and R. S. Mariano. Comparing Predictive Accuracy. Journal of Business and Economic Statistics, 13(3):134–144, 1995. [13] P. Fisher and M. Salmon. On evaluating the importance of nonlinearity in large macroe- conometric models. International Economic Review, 27(3):625–646, 1986. [14] Z. Griliches. A note on serial correlation bias in estimates of distributed lags. Economet- rica, 29(1):65–73, 1961. REFERENCES 440 [15] J. M. Hammersley and D. C. Handscomb. Monte Carlo Methods. Chapman and Hall, London, 1964. [16] J. M. Hammersley and K. W. Morton. A new Monte Carlo technique: antithetic variates. Proceedings of the Cambridge Philosophical Society, 52(3):449–475, 1956. [17] D. F. Hendry. Monte Carlo experimentation in econometrics. In Z. Griliches and M. D. In- triligator, editors, Handbook of Econometrics, vol. 2, volume 2, Amsterdam, 1984. North- Holland Publishing Co. [18] M. G. Kendall. Note on bias in the estimation of autocorrelation. Biometrika, 41(3- 4):403–404, 1954. [19] J. P. C. Kleijnen. Antithetic variates, common random numbers and optimal computer time allocation in simulation. Management Science, 21(10):1176–1185, 1975. [20] L. R. Klein. The Estimation of Distributed Lags. Econometrica, 26(4):553–618, 1958. [21] L. M. Koyck. Distributed Lags and Investment Analysis. North-Holland Publishing Co., Amsterdam, 1954. [22] S. Makridakis, A. Anderson, R. Carbone, R. Fildes, M. Hibon, R. Lewandowski, J. Newton, E. Parzen, and R. Winkler. The accuracy of extrapolation (times series) methods: Results of a forecasting competition. Journal of Forecasting, 1(2):111–153, 1982. [23] S. Makridakis and M. Hibon. Accuracy of forecasting:An empirical investigation (with discussion). Journal of the Royal Statistical Society, Series A, 142(2):97–145, 1979. [24] R. S. Mariano and B. W. Brown. Stochastic simulation, prediction and validation of non- linear models. In L. R. Kline and J. Marquez, editors, Economics in Theory and Practice: an Eclectic Approach. [25] F. H. C. Marriott and J. A. Pope. Bias in the estimation of autocorrelations. Biometrika, 41(3/4):390–402, 1954. [26] M. Nerlove. Distributed Lags and Demand Analysis for Agricultural and Other Commodi- ties. In U.S. D.A. Agricultural Handbook No. 141, Washington, 1958. U.S. Department of Agriculture. [27] A. D. Ridley. Combining Global Antithetic Forecasts. International Transactions in Oper- ational Research, 2(4):387–398, 1995. [28] A. D. Ridley. Optimal Weights for Combining Antithetic Forecasts. Computers & Industrial Engineering, 2(2):371–381, 1997. [29] A. D. Ridley. Optimal antithetic weights for lognormal time series forecasting. Computers & Operations Research, 26(3):189–209, 1999. REFERENCES 441 [30] A. D. Ridley and P. Ngnepieba. Antithetic time series analysis and the CompanyX data. Journal of the Royal Statistical Society A, 177(1):83–94, 2014. [31] A. D. Ridley, P. Ngnepieba, and D. Duke. Parameter Optimization for Combining Lognor- mal Antithetic Time Series. European Journal of Mathematical Sciences, 2(2):235–245, 2013. [32] B. D. Ripley. Stochastic Simulation. Wiley, New York, June, 1987. [33] L. Shi and K. Kapur. Quasi Feedforward and Feedback Control for Random Step Shift Disturbance Models. Quality Technology and Quantitative Management, 12(1):69–82, 2014. [34] L. Shi and K. Kapur. A Synthesis of Feedback and Feedforward Control for Stationary and Nonstationary Disturbance Models. Quality, Reliability, and Engineering International, 31(3):343–354, 2015. Appendix A: MSE Before Combining Substituting from (4), the total error in X̂ t , including purely random error and systematic error due to bias is X̂ t − X t = +Φ � X t−1 −µX � + § Cov(ǫt ,X t−1) Var(X t) ª� X t−1 −µX �− X t . Since µX , Φ and Cov(ǫt ,X t−1) Var(X t) are constant for a specified time series and model, the variance of the total error before combining is the MSE, given by MSE � σ2 � =Var � X̂ t − X t � =Var � µX +Φ � X t−1 −µX � + ¨ Cov � ǫt , X t−1 � Var � X t � � X t−1 −µX �− X t «� =   ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1  Var � X t �− 2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « Cov � X t , X t−1 � From (1), Cov � X t , X t−1 � = ΦVar � X t � +Cov � ǫt , X t−1 � . Therefore, MSE � σ2 � =   ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1  Var � X t � − 2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «¨ ΦVar � X t � + Cov � ǫt , X t−1 � Var � X t � Var � X t � « =   ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1− 2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « REFERENCES 442 × ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «� Var � X t � =   ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1− 2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � =  1− ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � (A1) Appendix B: MSE After Combining Consider the combined fitted values bX c,t = wbX t + (1−w) bX ′t , where bX t = µX +Φ � X t−1 −µX � + ¨ Cov � ǫt , X t−1 � Var � X t � « � X t −µX � (B1) and bX ′t = limsbX→0, p→0− n X + rX̂ X̂ p � sX̂/sX̂ p ��bX p t − bX p t �o . Since we are investigating the effect of changing variance, consider only the limit as p → 0−, and allow sx to retreat from 0 as follows bX c,t = wbX t + (1−w) n X + rX̂ X̂ p � sX̂/sX̂ p ��bX p t − bX p t �o , p→ 0−. (B2) For the case of lognormal X t , such that ln X t ∼ N(µ,σ), the expected values and standard deviations are E � X t � = exp � µ+ σ 2 2 � , E � X p t � = exp � pµ++ p2σ2 2 � and σX = exp ¦ µ+ σ 2 2 ©p exp (σ2)− 1,σX p = exp n pµ+ p2σ2 2 oq exp � p2σ2 �− 1, respectively. Re- placing population values with sample values and substituting in (B2), bX c,t =wbX t + (1−w)  X + rX̂ X̂ p � sX̂/sX̂ p � exp n ln X̂ + s2 2 op exp (s2)− 1 exp n pln X̂ + p2s2 2 oq exp � p2s2 �− 1 × � bX p t − exp � pln X̂ + p2s2 2 ��� , p→ 0− (B3) =wbX t + (1−w) � X +ϕ � p, bX t , s �� , p→ 0−, where ϕ � p, bX t , s � = rX̂ X̂ p exp n ln X̂ + s2 2 op exp (s2)− 1 exp n pln X̂ + p2s2 2 oq exp � p2s2 �− 1 � bX p t − exp � pln X̂ + p2s2 2 �� , p→ 0−. From Appendix D, lim p→0− ϕ � p, bX t , s � = exp � ln X̂ + s2 2 �n −ln X̂ + ln X̂ t o . REFERENCES 443 Therefore, as p→ 0−, (B3) becomes bX c,t = wbX t + (1−w) � X + exp � ln X̂ + s2 2 �n −ln X̂ + ln X̂ t o� . (B4) The combined mean square error MSEc= Var �bX c,t − X t � . Substituting for bX c,t from (B4) MSEc � σ2 � = Var � wbX t + (1−w) � X + exp � ln X̂ + s2 2 �� −ln X̂ + ln X̂ t �� − X t � where s2 is the variance of the logarithm ln bX t and w= § 1− Cov(ǫt ,X t−1) Var(X t) ª−1 (Ridley [29]). Substituting for bX t from (B1) MSEc � σ2 � =Var � w ¨ µX +Φ � X t−1 −µX � + ¨ Cov � ǫt , X t−1 � Var � X t � « � X t−1 −µX � « + (1−w) � X + exp � ln X̂ + s2 2 �� −ln X̂ + ln X̂ t �� − X t � Since X → µX as n→∞ and µX , Φ, and Cov(ǫt ,X t−1) Var(X t) are constant for a specified time series and model, then the post combining MSEc, is given by MSEc � σ2 � =Var � w ¨ ΦX t−1 + Cov � ǫt , X t−1 � Var � X t � X t−1 « + (1− w)exp � ln X̂ + s2 2 � ln X̂ t − X t � =w2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � + (1− w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � + Var � X t � + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « exp � ln X̂ + s2 2 � Cov � X t−1, ln X̂ t � (B5) − 2w ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « Cov � X t , X t−1 �− 2 (1−w)exp � ln X̂ + s2 2 � Cov � X t , ln X̂ t � From (1) Cov � X t , X t−1 � = � ΦVar � X t � +Cov � ǫt , X t−1 � = § Φ+ Cov(ǫt ,X t−1) Var(X t) ª Var � X t � . From the Ridley [29] antithetic time series theorem, ρln X̂ t ,X t = σp exp(σ2)−1 = Cov(ln X t ,X t)q Var(ln X t) q Var(X t) , and for a stationary time series, as n→∞, ln X̂ t → ln X t , Var � ln X̂ t � = s2, Var � X̂ t � = σ2 and s2→ σ2. So, Cov � ln X̂ t , X t � = σp exp(σ2)− 1 Ç Var � ln X t �Ç Var � X t � REFERENCES 444 = σ2/ q Var � X t � p exp(σ2)− 1 Var � X t � . Substituting for Cov � X t−1, X t � and Cov � ln X̂ t , X t � in (B5), MSEc � σ2 � =w2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � + (1− w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � + Var � X t � + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « exp � ln X̂ + s2 2 � Cov � X t−1, ln X̂ t � − 2w ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « Var � X t �− 2 (1−w)exp � ln X̂ + s2 2 � × σ 2/ q Var � X t � p exp(σ2)− 1 Var � X t � =w2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � + (1− w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � + Var � X t � + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « exp � ln X̂ + s2 2 � Cov � X t−1, ln X̂ t � (B6) − 2w ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t �− 2 (1−w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 Var � X t � =  �w2 − 2w � ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1− 2 (1−w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 + (1−w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � Var � X t � + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × exp � ln X̂ + s2 2 � Cov � ln X̂ t , X t−1 � Var � X t � � Var � X t � Appendix C: Per Unit Reduction in MSE Denoting the per unit reduction in MSE due to combining as PMSEc � σ2 � = MSE � σ2 �−MSEc � σ2 � MSE (σ2) REFERENCES 445 Substituting for MSE � σ2 � and MSEc � σ2 � from Appendix A and Appendix B. MSE � σ2 � =  1− ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t � MSEc � σ2 � =  �w2 − 2w � ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1− 2 (1−w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 + (1−w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � Var � X t � + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × exp � ln X̂ + s2 2 � Cov � ln X̂ t , X t−1 � Var � X t � � Var � X t � From which MSE � σ2 �−MSEc � σ2 � =  1− ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 Var � X t �− ��w2 − 2w � × ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 1− 2 (1−w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 + (1−w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � Var � X t � + 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × exp � ln X̂ + s2 2 � Cov � ln X̂ t , X t−1 � Var � X t � � Var � X t � =  − �w2 − 2w+ 1 � ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 2 (1−w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 − (1−w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � Var � X t � − 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × exp � ln X̂ + s2 2 � Cov � ln X̂ t , X t−1 � Var � X t � � Var � X t � =  −(w− 1)2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 2 (1−w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 − (1−w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � Var � X t � − 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « REFERENCES 446 × exp � ln X̂ + s2 2 � Cov � ln X̂ t , X t−1 � Var � X t � � Var � X t � . Substituting and canceling Var � X t � , PMSEc � σ2 � = A1 1− § Φ+ Cov(ǫt ,X t−1) Var(X t) ª2 , where A1 =− (w− 1)2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 2 (1− w)exp � ln X̂ + s2 2 � σ2/ q Var � X t � p exp(σ2)− 1 − (1−w)2exp � 2ln X̂ + s2 � Var � ln X̂ t � Var � X t � − 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × exp � ln X̂ + s2 2 � Cov � ln X̂ t , X t−1 � Var � X t � . Since ln X̂ → µ and s → σ, as n → ∞ then writing q Var � X t � , Var � X t � and Var � ln bX t � in terms of σ and simplifying, PMSEc � σ2 � = A2 1− § Φ+ Cov(ǫt ,X t−1) Var(X t) ª2 , A2 =− (w− 1)2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + 2 (1−w)exp � µ+ σ2 2 � × σ2 p exp(σ2)− 1 exp � µ+ σ 2 2 �p exp(σ2)− 1 − (1−w)2exp � 2µ+σ2 � σ2 exp � 2µ+σ2 � � exp(σ2)− 1 � − 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « exp � µ+ σ2 2 � × Cov � ln X̂ t , X t−1 � exp � 2µ+σ2 � � exp(σ2)− 1 � . REFERENCES 447 PMSEc � σ2 � = A3 1− § Φ+ Cov(ǫt ,X t−1) Var(X t) ª2 , A3 =− (w− 1)2 ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 + � 1− w2 � σ2 � exp(σ2)− 1 � − 2w (1−w) ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « Cov � ln X̂ t , X t−1 � exp � µ+ σ 2 2 � � exp(σ2)− 1 � . Substituting for w PMSEc � σ2 � = A+ B 1− § Φ+ Cov(ǫt ,X t−1) Var(X t) ª2 , (C1) where A=−   ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−1 − 1   2¨ Φ+ Cov � ǫt , X t−1 � Var � X t � «2 +  1− ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−2   × � σ2 � exp (σ2)− 1 � � and B =− 2 ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−1  1− ¨ 1− Cov � ǫt , X t−1 � Var � X t � «−1   ¨ Φ+ Cov � ǫt , X t−1 � Var � X t � « × Cov � X t−1, ln bX t � exp ¦ µ+ σ 2 2 ©� exp (σ2)− 1 � and Cov(ǫt ,X t−1) exp{2µ+σ2}(exp(σ2)−1) 6= 1. Substituting for Var � X t � PMSEc � σ2 � = C + D 1− § Φ+ Cov(ǫt ,X t−1) exp{2µ+σ2}(exp(σ2)−1) ª2 , (C2) where C =−   ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−1 − 1   2¨ Φ+ Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «2 REFERENCES 448 +  1− ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−2   σ2 � exp (σ2)− 1 � D =− 2 ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−1  1− ¨ 1− Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � «−1   × ¨ Φ+ Cov � ǫt , X t−1 � exp � 2µ+σ2 � exp (σ2)− 1 � « Cov � ǫt , X t−1 � exp ¦ µ+ σ 2 2 ©� exp (σ2)− 1 � . Appendix D: Limit as p→ 0− We wish to find lim p→0−, s→0 ϕ � p, bX t , s � = lim p→0− rbzbzp exp n ln bX + s2/2 op exp (s2)− 1 exp n pln bX + p2s2/2 oq exp � p2s2 �− 1 × h bX p t − exp � p · ln bX + p2s2/2 �i . Consider the Taylor expansion at p = 0 of exp(p) = 1+ p+ p2 2 + p3 6 + p4 24 + o � p4 � . (D1) Using (D1), we derive the Taylor expansion at p = 0 of exp(p2s2)−1= p2s2+ p4s4 2 + p6s6 6 +o � p6 � . Hence, 1q exp � p2s2 �− 1 = 1Ç p2s2 + p4s4 2 + p6s6 6 + o � p6 � = 1 s|p| Ç 1+ p2s2 2 + p4s4 6 + o � p4 � (D2) Using equation (D1), we derive the Taylor expansion at p = 0 of exp � p · ln bX + p2s2/2 � =1+ p · ln bX + p2s2/2+ 1 2 n p · ln bX + p2s2/2 o2 + o � p2 � =1+ p · ln bX + p2s2/2+ p2 2 · ln bX 2 + o � p2 � (D3) The Taylor expansion at p = 0 of �bX t �p is bX p t = 1+ p · ln bX + p2 2 · ln2 bX + o � p2 � (D4) REFERENCES 449 By subtracting (D3) from (D4), we obtain the Taylor expansion at p = 0 of n bX p t − exp � p · ln bX + p2s2/2 �o =p � −ln bX + ln bX t � (D5) + p2 2 � −ln bX 2 − s2 + ln2 bX t � + o � p2 � The Taylor expansion of ϕ � p, bX t , s � at p = 0 may now be obtained by multiplication of rbX bX p exp n ln bX+s2/2 op exp(s2)−1 exp n p·ln bX+p2s2/2 o and the expansions (D2) and (D5) as follows. ϕ � p, bX t , s � =rbX bX p exp � ln bX + s2/2 �p exp (s2)− 1 exp � p · ln bX + p2s2/2 � · 1 s|p| Ç 1+ p2s2 2 + p4s4 6 + o � p4 � × � p � −ln bX + ln bX t � + p2 2 � −ln bX 2 − s2 + ln2 bX t � + o � p2 �� . From Ridley [29] antithetic time series theorem, rbX bX p ↓ − sp exp(s2)−1 as p ↓ 0−. Also, limp→0− p |p| = −1, and, therefore, lim p→0−, ϕ � p, bX t , s � = exp � ln bX + s2 2 �� −ln bX + ln bX t � .