






































Microsoft Word - 00_tresc.docx


DYNAMIC ECONOMETRIC MODELS 
Vol. 10 – Nicolaus Copernicus University – Toruń – 2010 

Witold Orzeszko 
Nicolaus Copernicus University in Toruń 

Measuring Nonlinear Serial Dependencies Using  
the Mutual Information Coefficient† 

A b s t r a c t:  Construction, estimation and application of the mutual information measure have 
been presented in this paper. The simulations have been carried out to verify its usefulness to 
detect nonlinear serial dependencies. Moreover, the mutual information measure has been applied 
to the indices and the sector sub-indices of the Warsaw Stock Exchange. 

K e y w o r d s: nonlinearity, mutual information coefficient, mutual information, serial depen-
dencies.  

1. Introduction 
 Measuring relationships between variables is an extremely important area of 
research in econometrics. To this end the Pearson correlation coefficient is 
commonly used. However, the Pearson coefficient is not a proper tool for 
measuring nonlinear dependencies. Therefore, in the case of nonlinearity other 
methods must be used. The mutual information coefficient is one of the most 
important tools to detect nonlinear relationships. It comes from the information 
theory and is based on a concept of entropy. The mutual information coefficient 
may be applied to measure dependencies between two time series or serial de-
pendencies in a single time series. 

2. Measuring Nonlinear Dependencies in Time Series  
 There are various methods to measure nonlinear dependencies in time series 
(cf. Granger, Terasvirta, 1993; Maasoumi, Racine, 2002; Bruzda, 2004). One of 
the most important is the mutual information measure (MI hereafter), given by 
the formula: 

                                                 
† Financial support of Nicolaus Copernicus Univerity in Toruń for the project UMK 397-E 

is gratefully acknowledged. 



Witold Orzeszko 98

,
)()(

),(
log),(),(

21
∫∫ ⎟⎟

⎠

⎞
⎜⎜
⎝

⎛
= dxdy

ypxp
yxp

yxpYXI  (1) 

where ),( yxp  is a joint probability density function and )(1 xp  and )(2 yp  are 
marginal densities for random variables X and Y. 
 It can be shown that for all X and Y the measure ),( YXI  takes non-negative 
values and 0),( =YXI only if X and Y are independent.  

 It is convenient to define the mutual information coefficient, given by the 
expression: 

.1),( ),(2 YXIeYXR −−=  (2) 

 It can be shown that the mutual information coefficient has the following 
properties (cf. Granger, Terasvirta, 1993; Granger, Lin, 1994): 

1. 1),(0 ≤≤ YXR , 

2. 0),( =YXR ⇔ X and Y are independent, 

3. 1),( =YXR ⇔ )(XfY = , where f is  some invertible function, 

4. R is unaltered if X, Y are replaced by instantaneous transformations 
)(),( 21 YhXh ,  i.e.  ( ) ( ))(),(, 21 YhXhRYXR = ,  

5. if ( )YX ,  (or ( ))(),( 21 YhXh , where 1h  and 2h  are instantaneous) has 
a joint Gaussian distribution with correlation ),( YXρ , then 

),(),( YXYXR ρ= . 

 In the literature one can find several methods for estimating a value of 
),( YXI . Essentially, due to the technique of estimating the probability density 

functions in Equation 1, they can be divided into three main groups (cf. Dioni-
sio, Menezes, Mendes, 2003): 
− histogram-based estimators, 
− kernel-based estimators, 
− parametric methods. 

 The kernel-based estimators have many adjustable parameters such as the 
optimal kernel width and the optimal kernel form, and a non-optimal choice of 
those parameters may cause a large bias in the results. For the application of 
parametric methods one needs to know the specific form of the generating 
process (Dionisio, Menezes, Mendes, 2003)). Therefore a standard way is to 
estimate the densities by means of histograms (cf. Darbellay, Wuertz, 2000). 
 One can also define auto mutual information at lag k for a stationary dis-
crete-valued stochastic process nXXX ,...,, 21  as the mutual information be-
tween random variables tX  and ktX + : 



Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 99

.
)()(

),(
log),(),( ∑∑

+

⎟⎟
⎠

⎞
⎜⎜
⎝

⎛
=

+

+
++

t ktx x ktt

ktt
kttktt xPxP

xxP
xxPXXI  (3) 

 Since the process is stationary, ),( ktt XXI +  is independent of t and so we 
can refer to the mutual information at lag k, as )(kI  (Fonseca, Crovella, Sala-
matian, 2008).  
 This means that, the mutual information measure may be used to measure 
serial dependencies in a single time series as well. To this end, the past realiza-
tions of the investigated data X should be taken as the variable Y. 
 It should be emphasized that MI measures both linear and nonlinear depen-
dencies, so to identify serial nonlinear relationships, analyzed data must be pre-
filtered by an estimated ARMA-type model. 

3. Application of the Mutual Information Measure to Detect Serial 
Dependencies 

3.1. Simulated Data 

 The aim of the simulations was to verify, if the mutual information measure 
may be effectively applied to detect nonlinear serial dependencies. 
 The time series produced from five different generating models and two 
different sample sizes (with each of those models) were used in the simulations. 
This data was generated by Barnett et al. (1998) to compare the power of some 
popular tests for nonlinearity and chaos1. Specifically, these were: five time 
series of 2000 observations – M1, M2, M3, M4, M5 and five time series of their 
first 380 observations – M1s, M2s, M3s, M4s, M5s.  
 The investigated series were generated from the following models2: 
I) M1 – logistic map3: 

),1(57.3 11 −− −= ttt xxx  (4) 

II) M2 – GARCH(1,1) process: 

,ttt uhx =  (5a) 

,8.01.01 1
2

1 −− ++= ttt hxh  (5b) 

where 10 =h  and 00 =x . 
 

                                                 
1 The data was downloaded from the homepage of W.A. Barnett: http://econ.tepper.cmu.edu/ 

/barnett/Papers.html. 
2 In all cases, the white-noise disturbances – ut were sampled independently from the standard 

normal distribution. 
3 The logistic map with the parameter equaled to 3.57 generates chaotic dynamics. 



Witold Orzeszko 100 

III) M3 – Nonlinear Moving Average Process (NLMA): 
,8.0 21 −−+= tttt uuux  (6) 

IV) M4 – ARCH(1) process: 

,5.01 2
1 ttt uxx −+=  (7) 

V) M5 – ARMA(2,1) process: 
,3.015.08.0 121 −−− +++= ttttt uuxxx  (8) 

where 10 =x  and 7.01 =x . 
 In each case the mutual information measure was calculated for the raw 
series and for its residuals from the fitted ARMA model.  
 First, stationarity was verified using the Augmented Dickey-Fuller test. The 
null hypothesis of a unit root was strongly rejected for the all investigated data, 
except M5s. Thus, instead of M5s, the series of its first differences – M5s_diff 
was chosen for further research. 
 In Table 1 the ARMA models fitted to analyzed series are presented4. 

Table 1.  ARMA models for the generated series 
Series ARMA model Series ARMA model 

M1 White noise (EX=0.648) M1s White noise (EX=0.649) 
M2 White noise (EX=0.034) M2s White noise (EX=0.067) 
M3 White noise (EX= 0.007) M3s White noise (EX= 0.033) 
M4 White noise (EX= 0.011) M4s White noise (EX= 0.018) 
M5 ARMA(1,1) M5s_diff MA(1) 

 Next, the Ljung-Box test was applied to test if the residual series are white 
noise. The test confirmed that no investigated residuals contain linear depen-
dencies.  
 To estimate the mutual information measure the method proposed by Fraser 
and Swinney  (1986) was used5. This method is based on an analysis of the two-
dimensional histogram. Briefly speaking, it consists in covering the two-
dimensional plane containing pairs ( )tt yx ,  with rectangular partitions and cal-
culating frequencies of points in each partition. Next, Equation 1 is used, i.e. the 
calculated frequencies are estimators of the probability density functions and 
the integration is carried out numerically. 
 Let ki  denotes an estimated value of the mutual information measure be-
tween variables tX  and ktX − . Due to a purpose of the research, the key task is 
to verify the hypothesis of mutual information measure’s insignificance (i.e the 
hypothesis of independence). To this end, for each investigated series and for 
                                                 

4 The models were selected based on the Schwarz criterion. 
5 In the calculations the m-file created by A. Leontitsis was used. 



Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 101

each 10...,,2,1=k , the p-value was evaluated through bootstraping6 with 
00010 repetitions7. In Tables 2-6 the calculated values of ki  and the corres-

ponding p-values (at the bottom) are summarized. The p-values not larger than 
0.005 are bolded8.  

Table 2.  Values of ki for M1s and M1 

        k 
series 1 2 3 4 5 6 7 8 9 10 

M1s 1.6927 
0.0000 

1.6963 
0.0000 

1.6123 
0.0000 

1.7148 
0.0000 

1.5919 
0.0000 

1.6849 
0.0000 

1.5412 
0.0000 

1.6381 
0.0000 

1.5379 
0.0000 

1.6560 
0.0000 

M1 2.0139 
0.0000 

2.0090 
0.0000 

2.0064 
0.0000 

2.2520 
0.0000 

1.9981 
0.0000 

1.9991 
0.0000 

1.9940 
0.0000 

2.2737 
0.0000 

1.9891 
0.0000 

1.9891 
0.0000 

Table 3.  Values of ki for M2s and M2 

        k 
series 1 2 3 4 5 6 7 8 9 10 

M2s 0.0848 
0.9616 

0.1538 
0.0201 

0.1191 
0.3802 

0.1308 
0.1786 

0.1231 
0.3052 

0.1616 
0.0081 

0.1701 
0.0029 

0.1162 
0.4412 

0.1281 
0.2187 

0.1228 
0.3090 

M2 0.0541 
0.0053 

0.0562 
0.0025 

0.0477 
0.0808 

0.0488 
0.0536 

0.0492 
0.0451 

0.0509 
0.0227 

0.0541 
0.0052 

0.0461 
0.1303 

0.0449 
0.1868 

0.0334 
0.9315 

Table 4.  Values of ki for M3s and M3 

        k 
series 1 2 3 4 5 6 7 8 9 10 

M3s 0.1857 
0.0492 

0.1586 
0.3316 

0.1425 
0.6241 

0.1469 
0.5429 

0.1323 
0.8032 

0.1028 
0.9927 

0.1897 
0.0353 

0.1600 
0.3096 

0.1525 
0.4389 

0.1606 
0.2987 

M3 0.0725 
0.0000 

0.0658 
0.0001 

0.0307 
0.9634 

0.0429 
0.2065 

0.0309 
0.9599 

0.0383 
0.5426 

0.0372 
0.6274 

0.0404 
0.3724 

0.0389 
0.4868 

0.0456 
0.0976 

Table 5.  Values of ki for M4s and M4 

       k 
series 1 2 3 4 5 6 7 8 9 10 

M4s 0.1365 
0.2663 

0.1667 
0.0205 

0.1442 
0.1562 

0.1349 
0.2940 

0.1198 
0.6104 

0.1367 
0.2613 

0.1347 
0.2959 

0.1327 
0.3361 

0.1435 
0.1641 

0.1464 
0.1303 

M4 0.1053 
0.0000 

0.0472 
0.0051 

0.0363 
0.3383 

0.0379 
0.2324 

0.0286 
0.9261 

0.0344 
0.5058 

0.0370 
0.2866 

0.0475 
0.0039 

0.0368 
0.3074 

0.0344 
0.5059 

                                                 
6 Bootstrap without replacement (i.e. permutation) was performed. Bootstrapped p-values cor-

respond to a one-sided test. 
7 In this way, for each of the filtered series an expected distribution of MI(1) (i.e. the MI 

measure with k=1) was determined. Next, this distribution has led to evaluation of the p-value for 
each k=1,2,...,10.  

8 Note that the rejection of the null of ki insignificance for at least one k=1,2,...,10 implies the 
rejection of the hypothesis of serial independence. Therefore, adopting the value 0.005 for each k 
implies that the probability for a type I error (in the test of serial independence) is approximately 
5%. 



Witold Orzeszko 102 

Table 6.  Values of ki for M5s and M5 

       k 
series 1 2 3 4 5 6 7 8 9 10 

M5s 1.4787 
0.0000 

1.1206 
0.0000 

0.9817 
0.0000 

0.8640 
0.0000 

0.7505 
0.0000 

0.6895 
0.0000 

0.6344 
0.0000 

0.6310 
0.0000 

0.6173 
0.0000 

0.6070 
0.0000 

M5s_ diff 0.1390 
0.5519 

0.1658 
0.1199 

0.1288 
0.7509 

0.1438 
0.4542 

0.1496 
0.3452 

0.2012 
0.0039 

0.1642 
0.1351 

0.1297 
0.7340 

0.1161 
0.9125 

0.1387 
0.5560 

M5s_ diffMA 0.1224 
0.7971 

0.1584 
0.1595 

0.1225 
0.7942 

0.1242 
0.7668 

0.1444 
0.3745 

0.1391 
0.4816 

0.1624 
0.1193 

0.1510 
0.2584 

0.1495 
0.2821 

0.1474 
0.3179 

M5 1.7145 
0.0000 

1.3154 
0.0000 

1.0949 
0.0000 

0.9504 
0.0000 

0.8414 
0.0000 

0.7597 
0.0000 

0.6958 
0.0000 

0.6449 
0.0000 

0.5917 
0.0000 

0.5584 
0.0000 

M5ARMA 0.0422 
0.2714 

0.0375 
0.6530 

0.0417 
0.3103 

0.0412 
0.3438 

0.0355 
0.8012 

0.0396 
0.4685 

0.0419 
0.2963 

0.0486 
0.0398 

0.0434 
0.2030 

0.0397 
0.4640 

 In Tables 7-8 the results of nonlinearity detection carried out by the MI 
measure are summarized. 

Table 7.  Results of nonlinearity detection for the long series 
Series Serial dependencies Nonlinearity 

M1 YES YES 
M2 YES YES 
M3 YES YES 
M4 YES YES 
M5 YES NO 

Table 8.  Results of nonlinearity detection for the short series 
Series Serial dependencies Nonlinearity 
M1s YES YES 
M2s YES YES 
M3s NO NO 
M4s NO NO 

M5s_ diff YES NO 

 As it is clearly seen, the MI measure correctly identified each of the investi-
gated long series. In an application to the short series it led to erroneous conclu-
sions in the case of M3s and M4s. The obtained result is consistent with studies 
by other authors, i.e. it indicates that histogram-based estimators may be unreli-
able in a case of a small number of observations (e.g. Dionisio, Menezes, Men-
des, 2003). 

3.2. Stock Market Indices 

 In this section the indices and the sector sub-indices of the Warsaw Stock 
Exchange from 2.01.2001–15.04.2009 (2078 observations) were analyzed. For 
the each index, the three time series were investigated: daily log returns, residu-
als from their ARMA and ARMA-GARCH models. Investigation of the residu-
als from the ARMA model gives information, if dependencies are nonlinear. 
If so, the standardized residuals from the ARMA-GARCH model were ana-



Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 103

lyzed, to verify if this class of processes can capture nonlinear dynamics found 
in the investigated data9. The results of this analysis are presented in  
Tables 9-20. 

Table 9.  Values of ki for the WIG index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0458 
0.0000 

0.0444 
0.0003 

0.0605 
0.0000 

0.0612 
0.0000 

0.0486 
0.0000 

0.0518 
0.0000 

0.0350 
0.0338 

0.0365 
0.0153 

0.0522 
0.0000 

0.0559 
0.0000 

MA(1) 0.0412 
0.0010 

0.0455 
0.0000 

0.0549 
0.0000 

0.0632 
0.0000 

0.0427 
0.0002 

0.0500 
0.0000 

0.0379 
0.0074 

0.0313 
0.1530 

0.0552 
0.0000 

0.0566 
0.0000 

MA(1)-
GARCH(3,1) 

0.0458 
0.0225 

0.0498 
0.0033 

0.0336 
0.7074 

0.0395 
0.2254 

0.0359 
0.5142 

0.0306 
0.8964 

0.0302 
0.9124 

0.0352 
0.5738 

0.0328 
0.7700 

0.0309 
0.8823 

Table 10. Values of ki for the WIG20 index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0514 
0.0000 

0.0415 
0.0106 

0.0577 
0.0000 

0.0690 
0.0000 

0.0489 
0.0002 

0.0509 
0.0000 

0.0388 
0.0381 

0.0388 
0.0373 

0.0438 
0.0029 

0.0537 
0.0000 

MA(1) 0.0456 
0.0011 

0.0471 
0.0006 

0.0579 
0.0000 

0.0687 
0.0000 

0.0506 
0.0001 

0.0510 
0.0001 

0.0402 
0.0187 

0.0458 
0.0009 

0.0439 
0.0028 

0.0545 
0.0000 

MA(1)-
GARCH(3,1) 

0.0441 
0.0410 

0.0457 
0.0222 

0.0337 
0.6723 

0.0382 
0.2962 

0.0311 
0.8499 

0.0333 
0.6978 

0.0307 
0.8683 

0.0384 
0.2806 

0.0303 
0.8867 

0.0272 
0.9756 

Table 11. Values of ki for the mWIG40 index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0728 
0.0000 

0.0508 
0.0000 

0.0630 
0.0000 

0.0603 
0.0000 

0.0545 
0.0000 

0.0660 
0.0000 

0.0508 
0.0000 

0.0343 
0.0058 

0.0397 
0.0002 

0.0428 
0.0000 

AR(3) 0.0511 
0.0000 

0.0458 
0.0000 

0.0539 
0.0000 

0.0569 
0.0000 

0.0508 
0.0000 

0.0462 
0.0000 

0.0465 
0.0000 

0.0376 
0.0002 

0.0379 
0.0002 

0.0460 
0.0000 

AR(3)-
GARCH(1,2) 

0.0340 
0.0964 

0.0301 
0.3434 

0.0278 
0.5657 

0.0404 
0.0039 

0.0264 
0.6980 

0.0377 
0.0182 

0.0250 
0.8188 

0.0309 
0.2750 

0.0283 
0.5131 

0.0295 
0.3955 

Table 12. Values of ki for the sWIG80 index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0911 
0.0000 

0.0551 
0.0000 

0.0680 
0.0000 

0.0579 
0.0000 

0.0597 
0.0000 

0.0546 
0.0000 

0.0498 
0.0000 

0.0416 
0.0014 

0.0440 
0.0003 

0.0426 
0.0006 

ARMA(1.2) 0.0478 
0.0000 

0.0386 
0.0031 

0.0538 
0.0000 

0.0502 
0.0000 

0.0397 
0.0020 

0.0479 
0.0000 

0.0371 
0.0074 

0.0340 
0.0349 

0.0369 
0.0083 

0.0376 
0.0056 

ARMA(1,2)- 
GARCH(1,1) 

0.0268 
0.7878 

0.0300 
0.5014 

0.0367 
0.0616 

0.0282 
0.6646 

0.0258 
0.8545 

0.0345 
0.1451 

0.0255 
0.8722 

0.0295 
0.5471 

0.0278 
0.7014 

0.0309 
0.4072 

                                                 
9 The fit of all estimated models was positively verified using the Box-Ljung and the Engle 

tests. 



Table 13. Values of ki for the WIG-Banking index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0429 
0.0000 

0.0439 
0.0000 

0.0628 
0.0000 

0.0556 
0.0000 

0.0485 
0.0000 

0.0518 
0.0000 

0.0476 
0.0000 

0.0516 
0.0000 

0.0602 
0.0000 

0.0443 
0.0000 

MA(1) 0.0421 
0.0000 

0.0469 
0.0000 

0.0566 
0.0000 

0.0542 
0.0000 

0.0609 
0.0000 

0.0530 
0.0000 

0.0421 
0.0000 

0.0544 
0.0000 

0.0565 
0.0000 

0.0496 
0.0000 

MA(1)-
GARCH(1,2) 

0.0387 
0.0534 

0.0346 
0.2279 

0.0347 
0.2242 

0.0278 
0.8131 

0.0308 
0.5525 

0.0320 
0.4328 

0.0276 
0.8250 

0.0302 
0.6136 

0.0306 
0.5702 

0.0354 
0.1790 

Table 14. Values of ki for the WIG-Construction index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0525 
0.0000 

0.0301 
0.2070 

0.0415 
0.0009 

0.0400 
0.0016 

0.0365 
0.0119 

0.0451 
0.0001 

0.0460 
0.0001 

0.0326 
0.0823 

0.0321 
0.1004 

0.0451 
0.0001 

ARMA(2,1) 0.0336 
0.0145 

0.0386 
0.0003 

0.0428 
0.0000 

0.0387 
0.0003 

0.0350 
0.0064 

0.0365 
0.0022 

0.0391 
0.0002 

0.0221 
0.7270 

0.0320 
0.0311 

0.0481 
0.0000 

ARMA(2,1)- 
GARCH(1,1) 

0.0286 
0.5966 

0.0289 
0.5661 

0.0321 
0.2637 

0.0293 
0.5281 

0.0239 
0.9422 

0.0305 
0.4099 

0.0263 
0.8061 

0.0251 
0.8875 

0.0301 
0.4452 

0.0338 
0.1569 

Table 15. Values of ki for the WIG-Developers index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.1392 
0.0063 

0.1477 
0.0013 

0.1290 
0.0246 

0.1154 
0.1188 

0.1859 
0.0000 

0.1292 
0.0240 

0.1255 
0.0392 

0.1370 
0.0091 

0.1699 
0.0000 

0.1353 
0.0118 

ARMA(1,1) 0.1479 
0.0022 

0.1562 
0.0006 

0.1466 
0.0028 

0.1144 
0.1664 

0.1506 
0.0017 

0.1515 
0.0014 

0.1488 
0.0021 

0.1258 
0.0531 

0.1412 
0.0079 

0.1484 
0.0022 

ARMA(1,1)- 
GARCH(1,2) 

0.0928 
0.9250 

0.1147 
0.5124 

0.0999 
0.8324 

0.0929 
0.9245 

0.1195 
0.3976 

0.1153 
0.4980 

0.1189 
0.4135 

0.1199 
0.3861 

0.0869 
0.9708 

0.1241 
0.3014 

Table 16. Values of ki for the WIG-Food index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0544 
0.0000 

0.0361 
0.0000 

0.0449 
0.0000 

0.0473 
0.0000 

0.0446 
0.0000 

0.0311 
0.0037 

0.0314 
0.0028 

0.0337 
0.0003 

0.0305 
0.0052 

0.0342 
0.0002 

ARMA(1,1) 0.0371 
0.0000 

0.0365 
0.0000 

0.0418 
0.0000 

0.0433 
0.0000 

0.0358 
0.0000 

0.0270 
0.0055 

0.0310 
0.0003 

0.0409 
0.0000 

0.0230 
0.0601 

0.0298 
0.0007 

ARMA(1,1)-
GARCH(1,1) 

0.0311 
0.5233 

0.0340 
0.2766 

0.0338 
0.2873 

0.0347 
0.2198 

0.0239 
0.9738 

0.0309 
0.5448 

0.0281 
0.7944 

0.0281 
0.7938 

0.0311 
0.5252 

0.0362 
0.1404 



Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 105

Table 17. Values of ki for the WIG-IT index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0486 
0.0000 

0.0388 
0.0038 

0.0466 
0.0000 

0.0573 
0.0000 

0.0454 
0.0000 

0.0443 
0.0001 

0.0335 
0.0580 

0.0513 
0.0000 

0.0469 
0.0000 

0.0434 
0.0002 

AR(1) 0.0585 
0.0000 

0.0359 
0.0449 

0.0476 
0.0000 

0.0619 
0.0000 

0.0553 
0.0000 

0.0488 
0.0000 

0.0314 
0.2556 

0.0499 
0.0000 

0.0543 
0.0000 

0.0409 
0.0032 

AR(1)-
GARCH(1,1) 

0.0362 
0.0778 

0.0251 
0.8876 

0.0270 
0.7622 

0.0339 
0.1810 

0.0222 
0.9796 

0.0260 
0.8343 

0.0282 
0.6611 

0.0244 
0.9176 

0.0303 
0.4641 

0.0293 
0.5646 

Table 18. Values of ki for the WIG-Media index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0481 
0.0555 

0.0560 
0.0049 

0.0448 
0.1229 

0.0562 
0.0047 

0.0475 
0.0642 

0.0456 
0.1020 

0.0350 
0.6139 

0.0422 
0.2144 

0.0304 
0.8539 

0.0393 
0.3531 

MA(1) 0.0484 
0.1063 

0.0571 
0.0097 

0.0516 
0.0473 

0.0529 
0.0333 

0.0519 
0.0432 

0.0398 
0.4922 

0.0464 
0.1644 

0.0450 
0.2159 

0.0446 
0.2322 

0.0426 
0.3298 

MA(1)-
GARCH(1,1) 

0.0484 
0.1380 

0.0414 
0.4673 

0.0427 
0.3925 

0.0481 
0.1451 

0.0510 
0.0735 

0.0352 
0.8260 

0.0397 
0.5785 

0.0363 
0.7691 

0.0465 
0.2020 

0.0370 
0.7320 

Table 19. Values of ki for the WIG-Oil&Gas index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0825 
0.0185 

0.0780 
0.0503 

0.0761 
0.0733 

0.0711 
0.1660 

0.0862 
0.0076 

0.0658 
0.3335 

0.0562 
0.7510 

0.0685 
0.2422 

0.0829 
0.0166 

0.0667 
0.3009 

AR(2) 0.0816 
0.0203 

0.0652 
0.3279 

0.0619 
0.4671 

0.0824 
0.0183 

0.0820 
0.0195 

0.0863 
0.0062 

0.0573 
0.6711 

0.0823 
0.0184 

0.0878 
0.0043 

0.0771 
0.0493 

AR(2)-
GARCH(1,1) 

0.0451 
0.8837 

0.0611 
0.2362 

0.0493 
0.7406 

0.0573 
0.3809 

0.0716 
0.0368 

0.0573 
0.3787 

0.0524 
0.6043 

0.0652 
0.1240 

0.0592 
0.3055 

0.0472 
0.8178 

Table 20. Values of ki for the WIG-Telecom index 

        k 
series 1 2 3 4 5 6 7 8 9 10 

log returns 0.0467 
0.0072 

0.0395 
0.1307 

0.0429 
0.0369 

0.0687 
0.0000 

0.0440 
0.0234 

0.0393 
0.1405 

0.0417 
0.0579 

0.0469 
0.0062 

0.0419 
0.0518 

0.0514 
0.0007 

GARCH(1.3) 0.0311 
0.5186 

0.0340 
0.2752 

0.0338 
0.2834 

0.0347 
0.2234 

0.0239 
0.9693 

0.0309 
0.5411 

0.0281 
0.7826 

0.0281 
0.7821 

0.0311 
0.5207 

0.0362 
0.1448 

 The results summarized in Tables 9-20 indicate that evidence of serial de-
pendencies was found for the most investigated indices10. The same conclusion 
may be drawn for the residuals from the ARMA models, which means that the 
detected dependencies are nonlinear. In most cases the estimated ARMA-
GARCH models were able to capture these nonlinearities. Only in the case 

                                                 
10 The exception is the WIG-Oil&Gas index. In this case the obtained result is rather unusual, 

i.e. filtering data by the ARMA model caused the appearance of significance of the MI measure. 



Witold Orzeszko 106 

of WIG and mWIG40 indices there are reasons to believe that identified nonli-
nearity is not caused by an ARCH effect. 

References 
Barnett W. A., Gallant A. R., Hinich M. J., Jungeilges J. A., Kaplan D., Jensen M. J. (1998), 

A Single-blind Controlled Competition among Tests for Nonlinearity and Chaos, Journal 
of Econometrics, 82.1, 157–192. 

Bruzda J. (2004), Miary zależności nieliniowej w identyfikacji nieliniowych procesów ekono-
micznych (Measures of nonlinear relationship in identification of nonlinear economic 
processes), Acta Universitatis Nicolai Copernici, 34, 183–203. 

Darbellay G.A, Wuertz D. (2000), The entropy as a tool for analysing statistical dependencies 
in financial time series, Physica A, 287, 429–439. 

Dionisio A., Menezes R., Mendes D.A. (2003), Mutual Information: a Dependence Measure for 
Nonlinear Time Series, Working Paper, http://129.3.20.41/eps/em/papers/0311/ 
/0311003.pdf  (10.02.2010). 

Fonseca N., Crovella M., Salamatian K. (2008), Long Range Mutual Information, Proceedings of 
the First Workshop on Hot Topics in Measurement and Modeling of Computer Systems 
(Hotmetrics ’08), Annapolis. 

Fraser A.M., Swinney H.L. (1986), Independent Coordinates for Strange Attractors from Mutual 
Information, Physical Review A, 33.2, 1134–1140. 

Granger C. W. J., Terasvirta T. (1993), Modelling Nonlinear Economic Relationship, Oxford 
University Press, Oxford. 

Granger C. W. J., Lin J-L. (1994), Using the Mutual Information Coefficient to Identify Lags 
in Nonlinear Models, Journal of Time Series Analysis, 15, 371–384. 

Maasoumi E., Racine J. (2002), Entropy and Predictability of Stock Market Returns, Journal 
of Econometric”, 107, 291–312. 

Orzeszko W. (2010), Detection of Nonlinear Autodependencies Using Hiemstra-Jones Test, 
Financial Markets. Principles of Modeling, Forecasting and Decision-Making, eds. Milo 
W., Szafrański G., Wdowiński P., 157–170. 

Współczynnik informacji wzajemnej jako miara zależności nieliniowych 
w szeregach czasowych 

Z a r y s  t r e ś c i.  W artykule scharakteryzowano konstrukcję, estymację oraz możliwości zasto-
sowania współczynnika informacji wzajemnej. Przedstawiono wyniki symulacji, prowadzących 
do weryfikacji jego przydatności w procesie identyfikacji zależności nieliniowych w szeregach 
czasowych. Ponadto zaprezentowano wyniki zastosowania tego współczynnika do analizy indek-
sów Giełdy Papierów Wartościowych w Warszawie. 

S ł o w a  k l u c z o w e: nieliniowość, współczynnik informacji wzajemnej, mutual information, 
identyfikacja zależności.  




