








































© 2016 Nicolaus Copernicus University. All rights reserved.  

http://www.dem.umk.pl/dem 

D Y N A M I C  E C O N O M E T R I C  M O D E L S  
DOI: http://dx.doi.org/10.12775/DEM.2016.003  Vol. 16 (2016) 37−47 

Submitted November 30, 2016  ISSN (online) 2450-7067 

Accepted December 17, 2016 ISSN (print) 1234-3862 

Józef Stawicki
*
 

Using the First Passage Times in Markov Chain Model 
to Support Financial Decisions on the Stock Exchange 

A b s t r a c t. The purpose of this article is to present the possibilities of using such a tool as 

Markov Chain to analyse the dynamics of returns observed at the Warsaw Stock Exchange. 

Process analysis is the basis for decision-making with regard to the accepted horizon. Ex-

pected times for achieving specified states, understood as intervals of rates of return, in par-

ticular those describing negative rates of return, are extremely important. In this context, there 

is a possibility of determining easily the value at risk with the accepted probability. 

K e y w o r d s: Markov Chain, First passage times, Normal white noise, VaR. 

J E L Classification: C58; F47. 

Introduction 

 Markov Chain can be found a useful tool when describing the phenome-

non of changes in the financial market (stock listings, currency exchange 

rates, trading volume, etc.). This description may serve to identify the mere 

phenomenon, its character, or to perform comparative analysis of markets. 

Constructing Markov Chain model begins with a precise determination of 

states. In the case of analysis of the return rates process, the classes may be 

intervals in which the rate of return can be contained. Determination of these 

intervals is defined by the need of research or the consequences of decisions 

made based on the process that has been identified and described with 

                                                 
* Correspondence to: Józef Stawicki, Nicolaus Copernicus University, Faculty of Eco-

nomic Sciences and Management, 13A Gagarina Street, 87-100 Toruń, Poland, e-mail: 

stawicki@umk.pl. 



Józef Stawicki 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

38 

a relevant model. Another very important stage in the construction of 

Markov Chain model is the choice of an estimation method. Observation of 

long time series allows applying a method based on microdata, i.e., direct 

observation of a change of a class. Predictions derived based on the esti-

mated Markov Chain model are of particular significance. Predictions can 

serve individual or institutional investors, as well as institutions evaluating 

and supervising a given market. Comparing the behaviours of real stock 

market processes with classical ones (for instance, with Gaussian white 

noise) and comparing ergodic distributions is interesting both from a theo-

retical point of view and practice of decision making processes. This article 

aims to show how the Markov Chain model through the construction of pro-

cess states, understood as a range which may contain a particular indicator, 

can be helpful when describing the phenomenon treated as random and iden-

tifying the risk of an investment decision. 

 The simplicity of the process of finite Markov Chain and its natural in-

terpretation creates opportunities for popularising the proposed analyses. 

1. Markov Chain and Average Passage Times  
Between States 

1.1. Homogenous Markov Chain 

 The Markov process with a discrete time parameter and discrete phase 

space is called Markov Chain (Ching, Ng, 2006; Stawicki, 2004). It is de-

termined by a sequence of stochastic matrices of the following form: 

 
rrij tp


 )((t)P ,  (1) 

i.e., matrices with positive elements and satisfying additional requirements 

in the following form: 

 
j

ijit tp 1)( . (2) 

When designating with tD  the unconditional decomposition vector of  ran-

dom variable tY , that is:  

 rtttt ddd ,,, 21 D ,   where }Pr{ iYd tit  , (3) 

we determine the probability with which the process at time t reaches the 

phase state i. The elements of the vector tD  satisfy the conditions below: 



Using the First Passage Times in Markov Chain Model... 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

39 

0 itit d    (4) 

and  

1 
i

itt d . (5) 

The dependence between unconditional decompositions of random variables 

tY  and 1tY  is shown by the equation resulting from the theorem of total 

probability: 

)(1 ttt PDD   , (6) 

thus 





t

k

t k
1

0 )(PDD . (7) 

Matrices  
rrij tp


 )((t)P  reflect the mechanism of changes in the distribu-

tion of the analysed random variable tY  over time. 

The Markov Chain },{ NtYt   with the phase space }...,,2,1{ rS   is re-

ferred to as a homogeneous Markov Chain, if conditional probabilities )(tpij  

of transition from phase state i  to state j  in a time unit, which means in the 

period from )1( t  to t , are not dependent on the choice of time t , i.e., 

ijijt ptp  )( . (8)  

In the case of a homogeneous Markov Chain dependence (5) and (6) take the 

following form: 

PDD  1tt , (9) 

thus 

t
t PDD  0 . (10) 

For homogeneous Markov Chains described by means of transition matrix 

 ijpP , the expected first passage and return times can be designated. The 

matrix of these times is determined by the formula (Decewicz, 2011): 

                  
   ,  (11) 

where   is a fundamental matrix and   is an ergodic matrix. The elements of 

matrix         can be interpreted simply as the expected number of steps 



Józef Stawicki 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

40 

needed to achieve state   after abandoning state  .  If     then we obtain the 

expected time of the first return. 

1.2. Parameter Estimation of the Transition Matrix Mased  
on Microdata 

 Due to the nature of the studied phenomenon, microdata is understood as 

the observations of the object in subsequent units and registration of the state 

of the object in a specific unit. Observation of the state change in the period 

from 1t  to t allows applying a maximum likelihood estimator in the fol-

lowing form: 












T

t

i

T

t

ij

ij

tn

tn

p

2

2

)1(

)(

ˆ , 

where: 



 


sitation  adverse  in the0

statein  wasat time andstatein   was1at timeobject  thewhen1
)(

jtit
tnij

 






situation  adverse  in the0

statein wasat time  object was  when the1
)(

it
tni  

This estimator has the desirable properties of fitting, asymptotic unbiased-

ness and has an asymptotic normal distribution with the expected value: 

ijij ppE )ˆ( , 

and the variance: 









T

t

i

ijij

ij

tn

pp
p

2

)1(

)1(
)ˆvar( . 

2. The Return Rate Process – Comparison with Gaussian White 
Noise  

 Research on the rate of return as a stochastic process has been the subject 

of a vast number of publications. The fundamental question about the nature 

of this process mostly concerned the issue of the properties of Gaussian 

white noise. In the case of the generated process of the rate of return, it is 



Using the First Passage Times in Markov Chain Model... 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

41 

easy to obtain the property of normality for both the unconditional distribu-

tion and conditional distributions. The transition matrix for a Markov Chain 

being the model in this process takes the form:  





























0228.01359.03413.03413.01359.00228.0

0228.01359.03413.03413.01359.00228.0

0228.01359.03413.03413.01359.00228.0

0228.01359.03413.03413.01359.00228.0

0228.01359.03413.03413.01359.00228.0

0228.01359.03413.03413.01359.00228.0

P , 

where the states of this chain are defined by intervals σ. The rate of return 

may belong to one of the following states: 

).,2[

),2,[

),,0[

),0,[

),,2[

),2,(

6

5

4

3

2

1

























S

S

S

S

S

S

 

 This matrix is composed of ergodic vectors which are characterized by 

Gaussian distribution. Adoption of the intervals in accordance with the three-

sigma rule facilitates the comparison of the selected processes. This method 

of determining the states of the observed stock market processes will be 

maintained in further parts of the work.  
  To provide an empirical example, analysis was performed of returns of 

the WIG index as well as of the rate of return for the BUDIMEX company 

listed on the Warsaw Stock Exchange for the period from 02 February 2003 

to 11 August 2016. It provided a series of daily observations amounting to 

3475. Despite the many advantages of logarithmic returns, simple returns 

were analysed due to their clear interpretation. 

For the WIG index, the expected value and standard deviation in the exam-

ined period were as follows: 

E(X) = 0.000424, 

STD(X) = 0.0123. 

The matrix of transition probabilities takes the form:  

 



Józef Stawicki 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

42 



























 .

0526.02105.03553.03289.00395.00132.0

0266.00976.04053.03876.00533.00296.0

0214.00885.03842.04034.00833.00192.0

0133.00951.04162.03515.01013.00226.0

0350.01051.03408.03248.01306.00637.0

0565.01321.02736.02642.00849.01887.0

P . 

When comparing this matrix with the transition matrix for Gaussian white 

noise, hypothesis    assuming the equality of matrices must be rejected 

(Decewicz, 2011, p. 51). Test 67.2872 P  relative to 67.5305,0,30
2  . 

 The comparison of the ergodic distribution for such a design of the chain 

with the process of Gaussian white noise is presented in Figure 1. 

 

Figure 1. The comparison of white noise with the ergodic distribution of the rate of 

return obtained from the WIG index  

For the rate of return obtained from BUDIMEX, the expected value and 

standard deviation in the examined period were as follows: 

E(X) = 0.000787, 

STD(X) = 0.0224. 

The matrix of transition probabilities takes the following form: 

 

0 

0,05 

0,1 

0,15 

0,2 

0,25 

0,3 

0,35 

0,4 

0,45 

 -3σ  -2σ  -σ  σ  2σ  3σ 

Gaussian WN 

WIG 



Using the First Passage Times in Markov Chain Model... 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

43 





























0841.01495.01869.03832.01589.00374.0

0459.01311.02787.04295.00918.00230.0

0459.00897.03672.04389.00640.00128.0

0174.00622.03507.04485.01011.00201.0

0505.01325.02808.03817,01199.00347.0

1266.01139.02911.02152.01013.01519.0

P . 

When comparing this matrix with the transition matrix for Gaussian white 

noise, hypothesis    assuming the equality of matrices must be rejected 

(Decewicz, 2011, p. 51). Test 87.3752 P  relative to 67.5305,0,30
2  . 

Comparing the different rows of the matrix, i.e., comparing the conditional 

distributions, it must be noted that in the case each of the conditional distri-

bution must be rejected. For 05.0  and the fifth order the table value 

07.1105,0,5
2  .  

The comparison of the ergodic distribution for such a design of the chain 

with the process of Gaussian white noise is presented in Figure 2. 

 

 
 Figure 2. The comparison of Gaussian white noise with the ergodic distribution of 

the rate of return obtained from the BUDIMEX company index 

0 

0,05 

0,1 

0,15 

0,2 

0,25 

0,3 

0,35 

0,4 

0,45 

0,5 

 -3σ  -2σ  -σ  σ  2σ  3σ 

Gaussian WN 

BUDIMEX 



Józef Stawicki 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

44 

3. First Passage Times  

 Analysis of the expected times of transition from state i to state and j is 

extremely important from the point of view of an individual or institutional 

investor. The investor is able to determine the horizon for the portfolio being 

constructed or decide about the position that should be taken. 

 Calculations were made using the WinQSB package. 

 The matrix for the WIG index takes the following form: 





























7.451.96.27.22.127.39

9.463.105.26.29.111.39

1.474.106.26.26.114.39

5.473.105.27.24.112.39

3.461.107.28.20.116.37

1.458.99.20.36.116.32

M . 

It is worth noting that the expected passage times from any state to the state 

which is characterized by a value greater than twice the standard deviation 

are very large and in excess of one month. It should be noted, however, that 

negative returns are faster achieved than non-negative ones. Somewhat dif-

ferent is the situation of the single company tested. The analysed rate of 

return for BUDIMEX was characterized by the following first passage times 

matrix:   





























4.320.116.35.24.107.49

9.334.112.34.23.115.50

7.349.119.23.26.111.51

9.342.120.33.22.117.50

7.333.112.35.29.109.49

5.304.113.39.21.119.43

M . 

Here, the expected passage times to the class of rates lower than twice the 

standard deviation are larger than the expected times of the first passage 

times to the state with a rate of return higher than twice the standard devia-

tion. 



Using the First Passage Times in Markov Chain Model... 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

45 

 4. Determination of VaR using Markov Chain 

 Quantification of risk usually occurs by determining the measure known 

as VaR. There are many ways of determining the value at risk depending on 

the model which is used by analysts or investors. Literature on this subject is 

really abundant. This article  proposes to use the Markov Chain methodol-

ogy to determine VaR. When referring to the classic definition of value at 

risk (Osińska, 2006; Doman, Doman, 2009), one should pay attention to the 

fact that the market value is linked to the present rate of return and the hori-

zon, i.e., the accepted time interval. These two elements are exposed in 

a Markov Chain. One takes the form of state as a range of the rate of return, 

the other is linked to the time aggregation (chains based on daily, weekly, 

and monthly data) and the expected time of reaching a specified state for the 

first time. 

 The idea is to construct states appropriately including the state of risk 

),,(1 VaRS   

as well as the state which contains the rate of return at the present time T 

),[ yxSB  . State 2S  can be determined in the form )0,[2 VaRS   as long as 

the current value of the rate satisfies the condition 0x . This structure state 

of state 2S  is not necessary and may be determined as ),[2 zVaRS   provid-

ing zx  . 

 Value at risk is determined empirically by changing the range of state 1S  

and thus at the same time the interval of state 2S  until the time when the 

transition probability 1Bp  in matrix P reaches the assumed level of risk. 

 Analysis of the transition matrix of the BUDIMEX company allows to 

determine VaR in accordance with the above manner. It was assumed that 

the last observed rate of return was contained in the interval 

)01.0;0.0[4  SSB . The rate of return as of 8
th
 of November, 2016 was  

r = 0.004149. 

Other classes were determined in as follows:  



Józef Stawicki 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

46 

).;03.0[

),03.0;01.0[

),01.0;0[

),0;01.0[

),01.0;[

),;(

6

5

4

3

2

1













S

S

S

S

VaRS

VaRS

 

 The current state, i.e., state BS , is state 4S .  

 The transition matrix allowing 05.01 Bp  is the following matrix: 

,

1564.01673.01964.01418.02545.00836.0

0835.01967.03010.01699.02072.00.0417

0639.01957.03069.02091.01751.0

0456.01626.03191.02219.01900,00608.0

0687.02311.02456.01944.01915.00687.0

1729.01729.01822.01122.02290.01308.0




























0.0494

P  

for which state )03.0;(1 S .  That means that VAR = –0.03 . 

The matrix determined in that way has also interesting expected times of 

achieving individual states. This matrix takes the following form: 





























6.123.51.48.57.41.17

7.132.56.36.50.5

0.142.56.33.52.5

2.144.55.33.51.55.17

9.130.58.34.51.54.17

3,123.51.49.58.42.16

17.9

17.8
M . 

The expected time of the first attainment of the state of risk from the present 

state is relatively high and amounts to 17.8 days. 

Conclusions 

 The proposals do not exhaust all the possibilities offered by the tool in 

the form of Markov Chain. A comparison of the processes with Gaussian 

white noise showed a considerable discrepancy. This only confirms the re-

sults already described by the subject literature.  



Using the First Passage Times in Markov Chain Model... 

DYNAMIC ECONOMETRIC MODELS 16 (2016) 37–47 

47 

Especially important is the proposal of determining VaR with a very simple 

interpretation in the Markov Chain model. The result in the form of expected 

time of achieving the appropriate states is significant. This applies to both 

processes where states were constructed by standard deviation and to the 

construction of states in search of value at risk. The above results are an 

incentive for undertaking further research. In particular, this concerns the 

influence of aggregation of states in Markov Chain on the results presented 

in the work. 

References   

Osińska, M.  (2006), Ekonometria finansowa, Polskie Wydawnictwo Ekonomiczne, Warsza-

wa.  

Doman, M., Doman, R. (2009), Modelowanie zmienności i ryzyka,Wolter Kluwer Polska, 

Kraków. 

Decewicz, A. (2011), Probabilistyczne modele badań operacyjnych, Oficyna Wydawnicza 

SGH, Warszawa. 

Podgórska, M., Śliwka, P., Topolewski, M., Wrzosek, M. (2002), Łańcuchy Markowa w teorii 

i w zastosowaniach, Oficyna Wydawnicza SGH, Warszawa. 

Ching, W., Ng, M.,K. (2006) Markov Chains Models, Algorithms and Applications, Springer 

Science+Business Media. 

Stawicki, J. (2004), Wykorzystanie łańcuchów Markowa w analizie rynku kapitałowego, 

Wydawnictwo UMK, Toruń.  

Wykorzystanie oczekiwanych czasów pierwszego przejścia  

i powrotu w modelu łańcuchów Markowa  

do wspomagania decyzji finansowych na giełdzie 

Z a r y s  t r e ś c i. Artykuł prezentuje możliwość wykorzystania narzędzia jakim są łańcuchy 

Markowa do analizy dynamiki stóp zwrotu obserwowanych na GPW. Analiza procesu jest 

podstawą podejmowania decyzji w zadanym horyzoncie. Oczekiwane czasy osiągnięcia 

zadanych stanów, w szczególności opisujących ujemne stopy zwrotu, jest niezwykle ważne. 

W tym kontekście pojawia się możliwość łatwego wyznaczania wartości narażonej na ryzyko 

z zadanym prawdopodobieństwem.  

S ł o w a  k l u c z o w e: Łańcuchy Markowa, oczekiwane czasy pierwszego przejścia i po-

wrotu, gaussowski biały szum, VaR. 




