© 2017 Nicolaus Copernicus University. All rights reserved. http://www.dem.umk.pl/dem D Y N A M I C E C O N O M E T R I C M O D E L S DOI: http://dx.doi.org/10.12775/DEM.2017.010 Vol. 17 (2017) 161−176 Submitted November 29, 2017 ISSN (online) 2450-7067 Accepted December 28, 2017 ISSN (print) 1234-3862 Barbara Będowska-Sójka * Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland A b s t r a c t. This paper empirically investigates various approaches to model time-varying systematic risk on the Polish capital market. A plenty of methods is examined in the developed markets and the Kalman filter approach is usually indicated as the best method for estimation of time-varying beta. However, there exists a gap in the studies for the emerging markets. In the paper we apply weekly data of fifteen stocks listed on the Warsaw Stock Exchange from banking and informatics sector. The sample starts at the beginning of 2001 and ends in 2015 including the hectic crisis period. We estimate beta within few competing approaches: two MGARCH models, BEKK and DCC, unobserved component model, and static beta from linear regression. All beta estimates are compared in the securities market line framework. We find that unobserved component beta together with beta from DCC model have higher predictive accuracy than beta from BEKK model or static beta. The beta estimates are positively correlated within the industry and negatively correlated for stocks from different sectors. Finally, the prediction of beta coefficients are more accurate for stocks from banking sector than for IT companies. K e y w o r d s: BEKK; DCC; Kalman filter; MGARCH; time-varying beta. J E L Classification: G15; Q47. * Correspondence to: Barbara Będowska-Sójka, Poznań University of Economics and Business, Faculty of Informatics and Electronic Economy, al. Niepodległości 10, 61-857 Poznań, Poland, e-mail: barbara.bedowska-sojka-ue.poznan.pl. This paper was presented at Forecasting Financial Markets and Economic Decision Making FindEcon 2016 in Łódź, 18th OxMetrics User Conference 2016 in London and Econometric Research in Finance Workshop 2016 in Warsaw. I would like to thank two anonymous reviewers for their valuable comments and suggestions. The usual disclaimer applies. 1 The other possible specifications are presented in Kurach and Stelmach (Kurach and This paper was presented at Forecasting Financial Markets and Economic Decision Making FindEcon 2016 in Łódź, 18th OxMetrics User Conference 2016 in London and Econometric Research in Finance Workshop 2016 in Warsaw. I would like to thank two anonymous reviewers for their valuable comments and suggestions. The usual disclaimer applies. https://orcid.org/0000-0001-5193-8304 Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 162 Introduction According to capital asset pricing model, CAPM, the beta as the measure of the systematic risk, is the only risk factor important for investors (Andersen, et al., 2006). The estimation and prediction of beta value is relevant for investment decisions as well as for measuring the performance of fund managers (e.g. through the Treynor ratio), Betas are also strongly required in asset pricing, portfolio selection, asset allocation and risk management (Choudhry and Wu, 2008). Moreover, investors need a good approximation of beta as it allows to measure the cost of capital. The important question in the literature is how to estimate true latent beta coefficient. In the classic CAPM, we assume that beta is a constant measure of systematic risk. However, a vast of the literature devoted to examination of the beta stability shows a considerable evidence against this assumption (Andersen et al., 2006; Brooks et al., 1998; Faff et al., 2000; Huang and Litzenberger, 1988; Jagannathan and Wang, 1996; Menchero et al., 2016). The static CAPM loses in favor of the conditional version of CAPM with time-varying betas (Campbell et al., 1997), which accounts for the fact that betas and expected returns vary over business cycle and “depend on the nature of the information available at any given point in time” (Jagannathan and Wang, 1996). A great number of empirical studies use different approaches to estimate beta, all having some advantages and drawbacks. The estimates from different models usually differ substantially. The most common approaches are: linear regression where beta is assumed to be stable (Dębski, et al., 2014), multivariate GARCH models (MGARCH), that are employed to estimate the time-varying beta (Brooks et al., 1998), realized betas derived from realized volatility introduced by Andersen and Bollerslev (1998) and developed in further works (Andersen et al., 2001; Andersen et al., 2010; Andersen et al., 2006; Hajizadeh et al., 2012), or beta estimated in the rolling window within a linear regression. Another and very promising approach to estimate time-varying beta is the Kalman filter technique (Brooks et al., 1998; Kurach and Stelmach, 2014; Lie et al., 2000). In few works the Kalman filter is found to perform better than the GARCH specifications (Brooks et al., 1998; Lie et al., 2000). Several papers examine the beta coefficients on the Warsaw Stock Exchange. Dębski et al. (2014) study the impact of sampling frequency of the beta estimates, Kurach and Stelmach (Kurach and Stelmach, 2014) focus on different behavior of sector beta that are estimated with Kalman filter Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 163 approach, while Dębski et al. (2016) examine the stability of the beta parameters over bull and bear market for 134 largest companies. The purpose of our paper is to compare beta coefficients obtained from different parametric methods. We focus on data of weekly frequency for fifteen stocks quoted on the Warsaw Stock Exchange in two sectors, banking and IT. According to FTSE Russell Country Classification the Warsaw Stock Exchange is perceived as an advanced emerging market (FTSE, 2016; Wyman, 2016). The stocks in our sample are listed through the relatively long period of time as for the non-developed market. In the paper we estimate time-varying beta from MGARCH models and use Kalman filter for unobserved component model. We examine how methods, that have been already used in the developed markets, work when applied to data from the Warsaw Stock Exchange. Two step agenda is used: first we calculate time- varying beta coefficients with different methods for each stock separately, and second we employ a procedure to asses which estimate is the closest to the true unobservable beta. This examination is done in the framework of Securities Market Line, and as such is similar to approach presented in Choudhry and Wu (2008). We find that beta estimates from Kalman filter together with beta estimates from DCC models have the highest accuracy. In our sample the beta estimates within one industry are highly positively correlated, while beta estimates from different sectors are characterized by negative correlation coefficients. We also find that in-sample errors are lower in case of banking companies when comparing to IT stocks. The rest of the paper is as follows: in Section 1 the data are described, Section 2 is devoted to model specification, in Section 3 we show how the different beta estimates are compared, in Section 4 the empirical results are described and Section 5 concludes. 1. Data Our sample data consists of prices of 15 stocks quoted on the WSE constantly from the beginning of the 2000 till the end of 2015. They are representing two sectors: banking and informatics. These two sectors have the biggest number of stocks representatives quoted constantly in the whole period of the study. The price data are obtained from Stooq database (www.stooq.pl), and they are adjusted for dividends and splits. These stocks are listed together with their full names, tickers, industry, size category and capitalization in Table 1. Generally, banks are big companies, whereas IT stocks belongs to different size groups. For the approximation of the market Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 164 portfolio we use WIG index which comprises all companies listed on the Warsaw Stock Exchange (WSE) Main List. In the study we also use risk free rate that is calculated as a mid-quote of 1 month WIBOR and WIBID rates (that are the Polish counterparts of LIBOR and LIBID rates), Table 1. The list of the stocks included in the sample Company name Ticker Sector Firm size (category) Firm size (in mln euro) HANDLOWY SA BHW Banking big 2256 ING SA ING Banking big 4746 MBANK SA MBK Banking big 3204 MILLENIUM SA MIL Banking big 1423 PEKAO SA PEO Banking big 7464 BZ WBK SA BZW Banking big 7088 BOS SA BOS Banking medium 152 ASSECOPOL SA ACP Informatics big 1012 CDPROJEKT SA CDR Informatics big 1134 COMARCH SA CMR Informatics big 320 SYGNITY SA SGN Informatics small 13 ELZAB SA ELZ Informatics medium 54 MACROLOGIC SA MCL Informatics small 16 SIMPLE SA SME Informatics small 7 LARK SA LRK Informatics very small 3 Note: The companies on the WSE are categorized according to their size measured by capitalization in the following manner: ‘big’ stands for capitalization higher than 250mln of euro, ‘medium’ is in the interval (50mln, 250mln), ‘small’ is in the interval (5mln, 50mln) and ‘very small’ stands for capitalization lower than 5mln euro. The last column reports capitalization of stocks at the end of 2016. The daily stock prices, index values and WIBID/WIBOR rates are aggregated into weekly data based on the last observation in the week. The whole sample consists of 887 weekly returns. In the further work we use the percentage logarithmic returns. The calculations and graphics are done in OxMetrics STAMP7 (Koopman et al., 2006), PcGive (Doornik and Hendry, 2006) and G@RCH (Laurent, 2013). 2. Beta Estimation In this section we briefly describe models that are used in the empirical part. The common factor for obtained measures is a time-varying feature of beta coefficient. We use two MGARCH specifications and Kalman filter in unobserved component models. Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 165 2.1. MGARCH Specifications: Scalar BEKK and DCC Models Within multivariate GARCH models the conditional beta are obtained. After preliminary estimations we consider two multivariate GARCH specifications: scalar BEKK model (Engle and Kroner, 1995), and dynamic conditional correlation model, DCC (Andersen et al., 2006; Engle, 2002). Each of these models is bivariate model with two equations, one for stock return, i R , and one for market portfolio return, M R : ),0(~ ... ... 1 22110 22110 ttt MtMtMMtMMMt ititiitiiit D eRRR eRRR He         (1) In the general BEKK(p,q) model the conditional covariance matrix, Ht, is described in the following way:       p j jjtj q i iititit 11 '''' EHEDeeDCCH (2) where Di and Ej are identity matrix multiplied by scalars. In our approach 1and1  pq . In the dynamic conditional correlation DCC model of Engle (Engle, 2002) the following specification is used:         P j jtjitit Q i i P j j Q i it tttt tttt 1111 1*1* )'()1( QzzSQ QQQR DRDH  (3) where ),...,,(diag 2/12/1 2 2/1 1 Nttt hhhD , kt h is the conditional variance described with univariate GARCH models, t z is the vector of standardized kt e , 2/1 / ktktkt hez  , t R is a matrix of time-varying correlation coefficients of t z , and * t Q is diagonal matrix in which elements are square roots of diagonal elements of matrix tQ . In our study in both MGARCH models, BEKK and DCC model, in the conditional mean equations (eq.1) ARMA(1,0) specification are considered. In BEKK model the conditional variance equations are modeled with Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 166 GARCH(1,1), whereas in DCC model the conditional variance is modeled with GJR-GARCH(1,1), that accounts for possible leverage effect (Andersen et al., 2007). In both models the conditional distribution of the model error terms is assumed to be Student t. As the conditional beta for a stock i is described as )var(/),cov( MtMtitit RRR , both estimates of conditional covariance and conditional variance come directly from MGARCH models fitted to the returns of an individual stock i R and returns of a market index M R , a proxy for the market portfolio. Both the conditional variance and the conditional covariance are provided in the matrix Ht. Finally, the time-varying beta series are calculated from the conditional covariances in MGARCH models. 2.2. Unobserved Component Model The second approach used in the paper is a state-space representation where the Kalman Filter is used. We apply the unobserved component model, UC, that is considered as a multiple regression model with time-varying coefficients. This specification is based on the theory of structural time series models presented in Harvey (1989), In the general form a time series is viewed as being decomposable into trend, seasonal, and cycle components. In the UC model it is assumed that near and far distant observations should not be given equal weight. For our purpose we use a local level model (random walk) with drift in the following way (Harvey 1989) 1 : TtRR ttMtittit ...,,1),0(~, 2   NID , (4) Tttttiit ...,,1),0(~, 2 1,    NID (5) where NID denotes Normally and Independently Distribute, and t and t are independent variables. Equation (4) is a measurement equation, whereas equation (5) is the transition equation. Within this specification, any shock to asset’ beta is persistent. The Kalman filter allows to obtain the time-varying beta. It should be noted that the specification of the UC model for weekly data might raise some concerns as this model assumes inter alia that the conditional variance of returns is homoscedastic (has no ARCH effect described in e.g. Bollerslev et al. (1994). However, this effect is pretty well 1 The other possible specifications are presented in Kurach and Stelmach (Kurach and Stelmach, 2014) or Będowska-Sójka (2015). Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 167 captured by BEKK and DCC models. Thus, if the weekly data are characterized by the ARCH effect, than the UC model is not appropriate 2 . Finally we also obtain static beta from the ordinary least square regression as a benchmark for the comparisons of time-varying beta estimates. 3. Comparisons of Different Betas There is no obvious benchmark for unobservable beta. Therefore some proxies must be introduced when attempting to establish the relative dominance of one method over another. In a similar manner to Choudhry and Wu (2008) after calculation of different beta measures, we compare them on the basis of the fit to the Securities Market Line, SML: )( ftMtitftit rRrR   (6) where ft r is a risk-free rate of return. With the estimates of time-varying betas, one easily calculates in-sample theoretical returns based on the market return and the risk-free rate of return that are actually observed. We assess the relative accuracy of time-varying beta estimates by comparing the theoretical return with the actual returns and calculating the residual,  : ititit RR ˆ (7) where it R is the actual return at time t, it R̂ is the theoretical return of stock i according to the SML, and Tt ,...,1 stands for the consecutive weeks. The comparison of beta estimates are based on the standard forecasting error measures: the lower the errors, the better in-sample beta approximation. We use two standard errors used most frequently in empirical studies (Menchero et al., 2016; Wang, 2009): Mean Absolute Error,    T t titi RR T MAE 1 ,, ˆ1 and Mean Square Error,    T t titi RR T MSE 1 2 ,, ) ˆ( 1 . Out of these two measures, MAE is less sensitive to outliers. We also consider median Relative Absolute Error, mRAE, calculated as a median of the distribution of ratios b ti b tititi RRRR ,,,, ˆ/ˆ  , 2 A possible solution to this issue was proposed by Rockinger and Urga (2001). Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 168 where b stands for benchmark model (the OLS beta in our case), The last error measure is perceived in the literature as a robust comparative measure of performance (Armstrong and Collopy, 1992). 4. Empirical Results We estimated four type of models for each of fifteen stocks. Due to the highly volatile measures of conditional beta within the first period, we excluded from the analysis first 52 observations and thus analysis starts from 2001. In Figure 1 we show beta estimates from different methods for two stocks, each representing different sector: BHW (banking) and CDR (informatics), The remaining graphics are available on request. While the overall dynamic of beta is similar across different approaches, beta estimates from unobserved component model seem to be most smoothed and therefore most stable, whereas the beta estimates from both MGARCH models are highly volatile. Figure 1. The estimates of beta for BZW and CDR Note: Beta estimates shown in the figure are the following: BEKK stands for conditional beta from MGARCH scalar BEKK models, DCC stands for conditional beta from MGARCH DCC model and UC stands for time-varying beta from unobserved component model. Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 169 We also calculate correlation coefficients for different beta estimates across the sample and find that these correlations are positive, medium strong and statistically significant. In case of both MGARCH (BEKK and DCC) models the correlation coefficient is on average 0.63, for BEKK and UC models the correlation accounts for 0.68, while for DCC and UC models the correlation is equal to 0.59. Table 2 reports the mean, minimum and maximum values of beta estimates from different methods. The differences in means obtained on the basis of different methods are not significantly different from each other. All OLS beta coefficients are significantly different from zero. In four out of fifteen cases OLS beta is not statistically different from 1. The minimum and maximum beta estimates show high variability within the sample. Table 2. Time Varying Beta Estimates for Polish Banking and Informatics Stocks BEKK DCC UN OLS beta Mean Min Max Mean Min Max Mean Min Max BHW 0.7814 –0.0887 1.5478 0.7485 0.2697 1.4857 0.8161 0.0970 1.2811 0.7670 ING 0.6881 –0.0638 1.7841 0.7248 0.4115 1.6610 0.7732 0.4315 1.1505 0.7954 MBK 1.2074 0.5514 2.0564 1.1883 0.7394 2.5867 1.2191 0.7114 1.8901 1.2638 MIL 1.2189 0.4439 2.0981 1.2388 0.6987 1.8624 1.2698 0.9030 1.8001 1.3310 PEO 1.1592 0.3874 1.7685 1.1189 0.5660 2.0766 1.1496 0.3451 1.5487 1.1367 BZW 0.9399 –0.2699 1.8446 1.0181 0.2614 2.0313 1.0453 0.1442 1.6265 1.0875 BOS 0.2717 –0.4900 1.1031 0.2742 –0.0956 0.6591 0.3536 0.0576 0.5768 0.3271 ACP 0.9367 –0.0665 2.2527 1.0216 0.3337 2.5907 1.0055 0.4907 2.0023 1.0496a CDR 0.9909 –0.6677 2.6589 1.0989 0.5225 2.2277 1.1818 0.9375 1.9225 1.2213 CMR 0.8684 0.0688 1.8595 0.9418 0.3789 2.1970 0.9473 0.2939 2.1340 1.0616a SGN 0.9736 –0.0142 2.2289 1.0125 0.2495 2.6802 0.9917 0.3650 2.1843 0.9963a ELZ 0.4080 –0.5330 1.7864 0.4407 0.1423 0.7134 0.4427 0.3515 0.6608 0.4612 MCL 0.5738 –0.2633 1.8075 0.6836 0.2787 1.3665 0.6460 0.1155 1.2151 0.6734 SME 0.4905 –0.0393 1.6444 0.5250 0.1732 0.8998 0.6088 0.2196 1.2768 0.6607 LRK 0.7959 –1.5195 4.4349 0.9401 0.3165 3.3889 0.9343 0.2236 1.6165 0.9578a Note: Mean is the arithmetic average of conditional beta obtained from BEKK, DCC and Kalman filter models, Min and Max stand respectively for the minimum and maximum value of the conditional beta estimates in the sample. OLS best stands for the point estimates of beta from the linear regression. All OLS beta are statistically different from 0; letter a is a subscript used to show the estimates that are not statistically significantly different from 1. To evaluate beta estimates, three different measures of errors based on in-sample fit to SML (eq. 6) are employed. Table 3 reports the error measures. In most cases the lowest value is observed for time-varying beta estimates from unobserved component model: MAE is the lowest in all cases, whereas the mRAE is the lowest in twelve out if fifteen cases. Next is the BEKK model with fourteen lowest forecasting errors, twelve in MSE and two mRAE. The linear beta obtains the lowest error measures in three cases, Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 170 all of them are MSE errors. The DCC model has got the only one lowest mRAE. Table 3. The measures of in-sample forecasting errors for different beta estimates MSE MAE mRAE MSE MAE mRAE MSE MAE mRAE BHW ING MBK BEKK 0.0322 0.0239 0.9816 0.0329 0.0229 0.9898 0.0359 0.0271 0.9853 DCC 0.0336 0.0241 0.9971 0.0344 0.0230 0.9915 0.0373 0.0273 0.9940 UC 0.0339 0.0236 0.9643 0.0344 0.0225 0.9873 0.0375 0.0264 0.9838 OLS 0.0328 0.0243 1 0.0334 0.0232 1 0.0362 0.0277 1 MIL PEO BZW BEKK 0.0468 0.0338 0.9954 0.0273 0.0209 0.9996 0.0326 0.0240 0.9884 DCC 0.0499 0.0341 0.9978 0.0288 0.0207 1.0000 0.0335 0.0242 0.9937 UC 0.0505 0.0334 0.9893 0.0287 0.0203 0.9920 0.0335 0.0233 0.9812 OLS 0.0488 0.0347 1 0.0278 0.0208 1 0.0321 0.0253 1 BOS ACP CDR BEKK 0.0413 0.0278 0.9983 0.0447 0.0318 0.9768 0.0725 0.0494 0.9973 DCC 0.0426 0.0274 0.9971 0.0495 0.0321 0.9830 0.0761 0.0491 0.9994 UC 0.0423 0.0274 0.9992 0.0496 0.0311 0.9753 0.0751 0.0490 0.9984 OLS 0.0418 0.0277 1 0.0481 0.0324 1 0.0747 0.0496 1 CMR SGN ELZ BEKK 0.0429 0.0308 0.9862 0.0550 0.0388 0.9998 0.0607 0.0396 0.9990 DCC 0.0454 0.0308 0.9876 0.0579 0.0387 0.9940 0.0610 0.0394 1.0007 UC 0.0454 0.0302 0.9767 0.0578 0.0380 0.9932 0.0603 0.0393 0.9986 OLS 0.0443 0.0316 1 0.0566 0.0388 1 0.0602 0.0395 1 MCL SME LRK BEKK 0.0679 0.0453 0.9996 0.0740 0.0504 0.9978 0.0821 0.0568 0.9909 DCC 0.0701 0.0451 1.0002 0.0779 0.0503 0.9993 0.0747 0.0565 0.9979 UC 0.0690 0.0448 0.9969 0.0778 0.0501 0.9949 0.0740 0.0558 0.9939 OLS 0.0688 0.0454 1 0.0770 0.0509 1 0.0733 0.0571 1 Note: OLS stands for beta obtained from the linear regression estimated for the whole sample, BEKK stands for conditional beta from MGARCH scalar BEKK models, DCC stands for conditional beta from MGARCH DCC model and UC stands for time-varying beta from unobserved component model with random walk. The lowest errors are underlined. If the predictive accuracy of betas across the sectors are compared, we notice that beta estimates in banking sector have almost 1.7 lower average MSE than the beta estimates in informatics sector. As the companies in the banking sector are generally bigger than these from IT, the beta estimates for big stocks seem to be more accurate than estimates for medium and small stocks. In Figure 2 we show the beta estimates from unobserved component model for all stocks. The behaviour of betas is similar within the industries: in case of stocks from banking sector (from BHW to BOS) the beta estimates increase over the whole sample period and specially in the beginning of Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 171 financial crisis in 2008. With respect to informatics stocks (from ACP to LRK), beta estimates generally decrease. Some exceptions in the overall tendency are recognized, e.g. beta in LRK is changing up and down as well as beta of BZW. The correlation matrix showing the interdependencies between beta coefficients estimated from UC model is presented in the Appendix. On the one hand, the correlation matrix shows that within banking sector the correlations between beta estimates are in majority positive and statistically significant. One exception is PEO, which shows negative or statistically insignificant coefficients with five out of six stocks. In case of IT sector the situation is similar – the correlations for stocks from this industry are positive and statistically significant with one exception, LRK, where for four out of six stocks the correlations are negative. Those two stocks, PEO and LRK, are respectively the biggest and the smallest in the sample. On the other hand, the correlations between stocks from two different sectors are in most cases negative and significant. Figure 2. The time-varying beta estimates from the UC model Note: Beta estimates shown in the figure are from unobserved component model. Each graph presents the beta coefficients for a single stock. As the differences between the predictive errors are rather small we decide to use the statistic that allows for comparing accuracy of the examined methods. We employ modified Diebold-Mariano statistic Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 172 (henceforth modDM) (Harvey et al., 1997), that examines if the forecasting precision differs significantly across the methods used. It is found to perform much better than the original Diebold-Mariano test for different forecast horizons, as well as in cases when the forecast errors are autocorrelated or have non-normal distribution (Choudhry and Wu, 2009), We calculate modified Diebold-Mariano statistic for MSE. These statistics are calculated in pairs, in which forecast errors come from one of the MGARCH models, Kalman filter or the OLS regression beta. In table 4 we show the results of the Diebold-Mariano statistic for each stock and each pairs of models (model1 and model2) separately. We reject the null of equal predictive accuracy at the 5% level. The statistic has a Student t distribution with 1T degrees of freedom, where T is a number of observations. In case of the negative values the errors from model 2 are lower than from model1, in case of positive values the opposite holds. Table 4. The comparison of predictive accuracy of models used in the study – the modified Diebold-Mariano statistics model1 BEKK BEKK BEKK DCC DCC UC model2 DCC UC OLS beta UC OLS beta OLS beta BHW –7.4207 –7.4223 –7.3172 –1.2862 2.5595 2.7477 ING –4.7595 –4.7775 –4.5831 –0.7200 3.6377 3.2197 MBK –0.3156 –0.4811 0.2765 –0.9381 3.5396 3.1355 MIL –4.1499 –4.5043 –3.7641 –0.4977 3.5911 2.0957 PEO 3.3009 3.2579 4.0393 0.6722 3.2991 2.0614 BZW –1.4723 –1.5255 –0.5930 –0.2289 5.9932 3.9648 BOS –9.8146 –9.4839 –9.7841 0.6072 2.3638 1.5911 ACP –4.8505 –4.8918 –4.6438 –0.7892 3.4923 3.8068 CDR –7.9989 –7.8383 –7.8351 1.4541 2.2466 1.2660 CMR –5.7058 –5.4700 –5.4087 –0.1598 3.8289 2.6390 SGN –8.0543 –7.9966 –7.8688 0.7554 2.8694 2.4824 ELZ –8.9527 –8.8164 –8.7934 1.7822 2.0485 0.3529 MCL –8.5644 –8.8225 –8.7245 1.8249 2.8946 1.5217 SME –9.0150 –8.8529 –9.0097 0.5598 2.8379 1.9662 LRK –10.4690 –10.0370 –9.9295 0.6386 2.6865 2.4141 Note: The table provides the modified Diebold-Mariano statistics. We compare MSE from two non- nested models, model1 and model2, for each stock separately. The negative values of the statistic indicate that predictive accuracy of model2 is better than of model1. The positive values of statistics indicate the opposite. The statistics in grey font have p-values higher than α=0.05 – in such cases the predictive accuracy of two models is equal. Based on the results of modified DM statistics we find that for our sample of stocks both beta from dynamic conditional correlation (DCC) models and unobserved component (UC) models provide better beta forecasts than BEKK models. Surprisingly, even point OLS estimate of beta Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 173 provides better forecasts than BEKK model. This result is consistent for all stocks with minor exceptions: first, in case of PEO stock BEKK model offers better forecasts than DCC and UC models. This company is one of the biggest and more liquid among those listed on the WSE. Second, in case of MBK and BZW the differences between errors are not statistically significant. Additionally, both DCC models and UC models have greater predictive accuracy than the point beta estimates, although in case of UC model the significant difference is observed only in 12 out of 15 stocks. We do not find any difference between forecasting accuracy of DCC model and UC model, although the latter model is usually presented as a winner in beta predictive horse races (Brooks et al., 1998; Choudhry and Wu, 2008; Lie et al., 2000). However, the result of the Diebold-Mariano tests might depend on the selection of the loss function, that is MSE. In case of stocks listed on the Warsaw Stock Exchange, which is the developed emerging market, the beta coefficients from Kalman filter method gives as good forecasts as from MGARCH DCC model. Conclusion In the modern investment theory time-varying beta concept replaces the static one. In the paper we compare differently estimated beta coefficient in securities market line framework for the stocks from the banking and IT sectors. These stocks have been listed constantly on the Warsaw Stock Exchange over the period 2000–2015. First of all, we find that contrary to the previous results presented in the literature (Dębski et al., 2016) the estimated beta coefficients in our sample are time-varying within the given period. We consider two MGARCH model specifications, DCC and BEKK model, the Kalman filter technique and the estimates from linear regression models. Our results show that beta estimated from the unobserved component models brings the most stable and smoothed betas. The estimates from BEKK models have often the lowest mean square error, while the estimates from Kalman filter in most cases offer the lowest mean absolute error as well as median relative absolute error. However, the comparison of predictive accuracy of methods used in the study shows that the errors obtained from the SML with beta estimated from BEKK models are significantly bigger than from the other models. The beta estimates of both DCC model and UC model fit better in terms of the SML than the estimates of beta from ordinary least squares. We do not find the evidence for statistical difference between the predictive accuracy of DCC model and UC models. Not surprisingly the Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 174 correlation coefficients for beta estimates within a sect or are positive, whereas between sectors are negative. Finally, when betas for stocks from different industries are compared, in-sample forecasts are more accurate for stocks from the banking sector than for stocks from the informatics sector. A natural extension of the presented study is to consider out-of-sample predictive accuracy of individual models for beta estimation. The results of such empirical exercise might be of practical interest for different groups of market professionals. References Andersen, T. G., Bollerslev, T. (1998), Answering the Skeptics: Yes, Standard Volatility Models Do Provide Accurate Forecasts. International Economic Review, 39(4), 885–905. Andersen, T. G., Bollerslev, T., Christoffersen, P., Diebold, F. X. (2007), Practical Volatility and Correlation Modeling for Financial Market Risk Management, Risks of Financial Institutions, 513–548, DOI: https://doi.org/10.3386/w11069. Andersen, T. G., Bollerslev, T., Christoffersen, P. F., Diebold, F. X. (2006), Volatility and Correlation Forecasting. Handbook of Economic Forecasting, 1, DOI: https://doi.org/10.1016/S1574-0706(05)01015-3. Andersen, T. G., Bollerslev, T., Diebold, F. X., Labys, P. (2001), The Distribution of Realized Exchange Rate Volatility, Journal of the American Statistical Association, 96, 42–55, DOI: https://doi.org/10.1198/016214501750332965. Andersen, T. G., Bollerslev, T., Diebold, F. X., Wu, G. (2006), Realized Beta: Persistence and Predictability, Advances in Econometrics, 2(05), 1–39, DOI: https://doi.org/10.1016/S0731-9053(05)20020-8. Andersen, T. G., Bollerslev, T., Meddahi, N. (2010), Realized Volatility Forecasting and Market Microstructure Noise, Journal of Econometrics, 160(1), 220–234, DOI: https://doi.org/10.1016/j.jeconom.2010.03.032. Armstrong, B. J. S., Collopy, F. (1992), Error Measures For Generalizing About Forecasting Methods: Empirical Comparisons By J. Scott Armstrong and Fred Collopy Reprinted with permission form, International Journal of Forecasting, 8(1), 69–80, DOI: https://doi.org/10.1016/0169-2070(92)90008-W. Będowska-Sójka Barbara. (2015), Unemployment Rate Forecasts: Evidence from the Baltic States, Eastern European Economics, 53(1), 57–67, DOI: https://doi.org/10.1080/00128775.2015.1033236. Bollerslev, T., Engle, R. F., Nelson, D. B. (1994), ARCH Models. In Engle, R. F. and McFadden D. (Eds.), The Handbook of Econometrics, 4, 2959–3038), Elsevier Science, Amsterdam. Brooks, R. D., Faff, R. W., McKenzie, M. D. (1998), Time-varying Beta Risk of Australian Industry Portfolios: A Comparison of Modeling Techniques, Australian Journal of Management, 23(1), 1–22. Campbell, J. Y., Lo, A. W., MacKinlay, A. C. (1997), The Econometrics of Financial Markets, Princeton University Press, Chichester. Choudhry, T., Wu, H. (2008), Forecasting Ability of GARCH vs Kalman Filter Method: Evidence from Daily UK Time-varying Beta, Journal of Forecasting, 27(8), 670–689, DOI: https://doi.org/10.1002/for.1096. Evaluating the Accuracy of Time-varying Beta. The Evidence from Poland DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 175 Dębski, W., Feder-Sempach, E., Świderski, B. (2014), Intervalling Effect On Estimating The Beta Parameter For The Largest Companies On The WSE, Folia Oeconomica Stetinensia, 14(2), 270–286, DOI: https://doi.org/10.1515/foli-2015-0018. Dębski, W., Feder-Sempach, E., Świderski, B. (2016), Beta Stability Over Bull and Bear Market on the Warsaw Stock Exchange, Folia Oeconomica Stetinensia, 16(1), 75–92, DOI: https://doi.org/10.1515/foli-2016-0006. Doornik, J. A., Hendry, D. F. (2006), Empirical Econometric Modelling using PcGive: Volume I, Timberlake Consultants Press, London. Engle, R. (2002), Dynamic Conditional Correlation – a Simple Class of Multivariate Garch Models, Journal of Business and Economic Statistics, 20(3), 339–350, DOI: https://doi.org/10.1198/073500102288618487. Engle, R. F., Kroner, K. F. (1995), Multivariate Simultaneous Generalized Arch, Econometric Theory, 11(1), 122–150. Faff, R. W., Hillier, D., Hillier, J. (2000), Time Varying Beta Risk: An Analysis of Alternative Modelling Techniques, Journal of Business Finance & Accounting, 27(5–6), 523–554, DOI: https://doi.org/10.1111/1468-5957.00324. FTSE. (2016), FTSE Country Classification. Hajizadeh, E., Seifi, A., Fazel Zarandi, M. H., Turksen, I. B. (2012), A Hybrid Modeling Approach for Forecasting the Volatility of S&P 500 Index Return, Expert Systems with Applications, 39(1), 431–436, DOI: https://doi.org/10.1016/j.eswa.2011.07.033. Harvey, D., Leybourne, S., Newbold, P. (1997), Testing the Equality of Prediction Mean Squared Errors, International Journal of Forecasting, 13(2), 281–291, DOI: https://doi.org/10.1016/S0169-2070(96)00719-4. Huang, C. F., Litzenberger, R. H. (1988), Foundations for Financial Economics, North Holland, New York. Jagannathan, R., Wang, Z. (1996), The Conditional CAPM and the Cross-Section of Expected Returns, Journal of Finance, 51(1), 3–53, DOI: https://doi.org/10.2307/2329301. Koopman, S. J., Harvey, A. C., Doornik, J. A., Shephard, N. (2006), STAMP 7: Structural Time Series Analyser and Modeller and Predictor, Timberlake Consultants Press, London. Kurach, R., Stelmach, J. (2014), Time-Varying Behaviour of Sector Beta Risk - the Case of Poland, Romanian Journal of Economic Forecasting, 17(1), 139–159. Laurent, S. (2013), G@RCH 7: Estimating and Forecasting ARCH Models. Timberlake Consultants Press, London. Lie, F., Brooks, R., Faff, R. (2000), Modelling the Equity Beta Risk of Australian Financial Sector Companies, Australian Economic Papers, 39(3), 301–311, DOI: https://doi.org/10.1111/1467-8454.00093. Menchero, J. G., Nagy, Z., Singh, A. (2016), Evaluating the Accuracy of Beta Forecasts, The Journal of Portfolio Management, 84–93. Rockinger, M., Urga, G. (2001), A Time Varying Parameter Model to Test for Predictability and Integration in the Stock Markets of Transition Economies, Journal of Business & Economic Statistics, 19(1), 73–84, DOI: https://doi.org/10.1198/07350010152472634. Wang, Y. H. (2009), Nonlinear Neural Network Forecasting Model for Stock Index Option Price: Hybrid GJR-GARCH approach, Expert Systems with Applications, 36(1), 564–570, DOI: https://doi.org/10.1016/j.eswa.2007.09.056. Wyman, O. (2016), Enhancing Liquidity in Emerging Market Exchanges, World Federation of Exchanges. Barbara Będowska-Sójka DYNAMIC ECONOMETRIC MODELS 17 (2017) 161–176 176 Appendix Table 5 The Spearman correlation matrix of beta coefficients from Kalman filter approach BHW ING MBK MIL PEO BZW BOS ACP BHW 1.00 0.67 0.66 0.57 -0.32 -0.06 0.98 -0.79 ING 0.67 1.00 0.83 0.77 0.03 0.30 0.67 -0.78 MBK 0.66 0.83 1.00 0.87 0.04 0.27 0.67 -0.68 MIL 0.57 0.77 0.87 1.00 -0.14 0.32 0.57 -0.65 PEO -0.32 0.03 0.04 -0.14 1.00 0.19 -0.27 0.31 BZW -0.06 0.30 0.27 0.32 0.19 1.00 -0.09 -0.36 BOS 0.98 0.67 0.67 0.57 -0.27 -0.09 1.00 -0.77 ACP -0.79 -0.78 -0.68 -0.65 0.31 -0.36 -0.77 1.00 CDR 0.08 -0.19 0.00 0.16 -0.35 -0.59 0.10 0.27 CMR -0.73 -0.27 -0.25 -0.08 0.15 0.01 -0.73 0.55 SGN -0.51 -0.62 -0.65 -0.47 -0.21 -0.53 -0.52 0.69 ELZ -0.79 -0.90 -0.83 -0.78 0.04 -0.34 -0.80 0.87 MCL -0.94 -0.71 -0.68 -0.65 0.35 -0.12 -0.92 0.87 SME -0.98 -0.68 -0.70 -0.59 0.27 0.00 -0.98 0.81 LRK 0.19 0.26 0.31 0.28 -0.08 -0.26 0.23 -0.08 ACP CDR CMR SGN ELZ MCL SME LRK BHW -0.79 0.08 -0.73 -0.51 -0.79 -0.94 -0.98 0.19 ING -0.78 -0.19 -0.27 -0.62 -0.90 -0.71 -0.68 0.26 MBK -0.68 0.00 -0.25 -0.65 -0.83 -0.68 -0.70 0.31 MIL -0.65 0.16 -0.08 -0.47 -0.78 -0.65 -0.59 0.28 PEO 0.31 -0.35 0.15 -0.21 0.04 0.35 0.27 -0.08 BZW -0.36 -0.59 0.01 -0.53 -0.34 -0.12 0.00 -0.26 BOS -0.77 0.10 -0.73 -0.52 -0.80 -0.92 -0.98 0.23 ACP 1.00 0.27 0.55 0.69 0.87 0.87 0.81 -0.08 CDR 0.27 1.00 0.23 0.54 0.15 -0.08 -0.02 0.55 CMR 0.55 0.23 1.00 0.53 0.41 0.66 0.77 0.31 SGN 0.69 0.54 0.53 1.00 0.74 0.58 0.57 0.27 ELZ 0.87 0.15 0.41 0.74 1.00 0.85 0.80 -0.22 MCL 0.87 -0.08 0.66 0.58 0.85 1.00 0.94 -0.21 SME 0.81 -0.02 0.77 0.57 0.80 0.94 1.00 -0.12 LRK -0.08 0.55 0.31 0.27 -0.22 -0.21 -0.12 1.00 Note: The table reports Spearman rank correlations for pairs of beta coefficients estimated with unobserved component model. The bolded values are statistically significant at α=0.05.