Microsoft Word - STAT090428-f - edited_corrected.doc SQU Journal For Science, 15(2010) 87-100 © 2010 Sultan Qaboos University 87 Filtering and M-ary Detection of Markov Modulated Mean Reverting Model Lakhdar Aggoun*, Mohamed Al-Lawati* and W.P. Malcolm** *Department of Mathematics and Statistics, College of Science, Sultan Qaboos University, P.O. Box 36, Postal code 123, Muscat, Sultanate of Oman, Email: laggoun@squ.edu.om. **Australian National University, Canberra Australia, Email: Malcolm@maths.anu.edu.au. نموذج مركوفي للفرق بين األسعار في سوق المال مالكولم. ب.و و األخضر عجون، محمد اللواتي هذا البحث يطور نموذج يحتوي على سلسلتان من نوع مرآوف تقوم بالتأثير على نموذج يقوم بدراسة حرآة للفرق :خالصة في سعار صيف أحداث غير معروفة لكن ذات تأثير على األ تقوم بتو السلسلتان من نوع مرآوف .ألسعار في سوق المال ابين .وتستعمل في هذا البحث طرق تغيير القياس لتقدير التوزيع الشرطي المتكرر. سوق المال ABSTRACT: In an earlier paper we developed a stochastic model incorporating a double-Markov modulated mean-reversion model. The model is based on an explicit discretisation of the corresponding continuous time dynamics. Here we discuss parameter estimation via the technique of M-ary detection. KEYWORDS: Double-Markov modulated mean-reversion model, Filtering, M-Ary detection, Continuous-Time Dynamics. mathematics subject classification. 60g35, 62m05, 62m20, 91b30, 91b70. 1. Introduction he model we developed in Malcolm et al. (2004) is a stochastic model incorporating a double Markov modulated mean reversion model. Unlike a price process the basis process can take positive or negative values. This model is based on an explicit discretization of the corresponding continuous time dynamics. In that model we suppose the mean reverting level in our dynamics as well as the noise coefficient can change according to the states of some finite-state Markov processes which could be the economy and some other unseen random phenomenon. In this paper we wish to discuss -ary detection for this model. The term -ary detection is used in Electrical Engineering to describe sequential hypothesis testing for more than two candidate model hypotheses. Here we are interested in model-parameter hypotheses. In effect our formulation is something like a discrete and finite version of the EM algorithm by Baum and Petrie (1966), Dempster et al. (1977) where, rather than considering an uncountable collection of model parameter sets in the space of all admissible models, we consider a finite collection in this space. T LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 88 We assume that we have a list of candidate models, from which to choose, describing the model dynamics over time. These candidate models will be denoted by , . Let be a simple random variable denoting a specific model, with states indexed by . We assume that is taking on values in the canonical basis of . We suppose is an indicator random variable such that , that is if and only if hypothesis holds. Here is the usual inner product. We shall be interested in computing the posterior probabilities , where denotes information contained in some observation process. It will be shown that this problem separates into a pure filtering component and a pure estimation component. In the context of -ary detection, this is known as the Separation Theorem (Poor 1988). This paper is organized as follows. In §2 & §3 we recall the model dynamics as well as the construction of a new probability measure under which all processes are independent. In §4 M-ary Detection Filters are derived. In §5 & §6 our results are adapted to continuous time dynamics. 2. Stochastic Dynamics All models are, initially, on the probability space ( )F PΩ, , . Write { 0 }uX X u t= , ≤ ≤ , for the basis (price difference) process. tX R∈ . Suppose L is a mean reversion level and Rα +∈ is the rate-parameter, that is, a parameter determining how fast the level L is attained by the process X . X has dynamics: 0 0 ( ) t t u tX X L X du Wα σ= + − + .∫ (2.1) Here W is a standard Wiener process, and Rσ ∈ . Remark 1. The dynamics at (2.1) exhibit a mean reversion1 character of the model when written in stochastic differential equation form: ( )t u tdX L X dt dWα σ= − + . (2.2) Ignoring the noise tdWσ , if tX L> then ( ) 0tL Xα − < , while if tX L< then ( ) 0tL Xα − > , and so the right side of is continually trying to reach the level L. Now suppose that parameters L and σ are stochastic and can switch between different levels 1 2 mL L … L, , , and 1 n…σ σ, , respectively. We assume here that these levels are determined by the states of two Markov chains Z and respectively. Without loss of generality, we take the state spaces of our Markov chains to be the canonical basis 1 2{ }mL e e … e= , , , of mR and the canonical basis 1 2{ }nS f f … f= , , , of nR respectively. 1 Modeling a mean reversion process is widely used in finance, for example in interest rates models such as the Vasicek Model. This class of models assumes an (static) average value will be attained, not unlike the notion of an equilibrium state, or steady state of a dynamical system in the physical sciences. FILTERING AND M-ARY DETECTION OF MARKOV 89 Write (2.3) (2.4) Write . Then (2.5) Here, and are martingale increments. The scalar-valued Markov processes taking values 1 mL … L, , and 1 n…σ σ, , , are obtained by (2.6) Here 1 2( )mL L … L ′= , , ,L , 1 2( )n…σ σ σ ′= , , ,S , 〈⋅,⋅〉 denotes an inner product and { }1 A denotes an indicator function for the event A . What also we wish to impose is that the two Markov chains Z and be not independent, that is, information on the behavior of one conveys some knowledge of the behavior of the other. More precisely, we assume the dynamics: (2.7) where js ir ⎛ ⎞ ⎜ ⎟,⎝ ⎠ =P p denotes a mn mn× matrix, or tensor, mapping into and Again 1k +M is a martingale increment. The dynamics at (1) take the form (2.8) Remark 2. We defined Z and as inherently discrete-time. Here, we "read" Z and as the output of a sample and hold circuit, or CADLAG processes. • What we wish to do now, is discretise the dynamics at (8) and then compute a corresponding filter and detector. LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 90 • We will use an Euler-Maruyama discretisation scheme to obtain discrete-time dynamics, although many other schemes can be used; see, for example, Numerical Solution of Stochastic Differential Equations by Kloeden and Platen (1992). For all time discretisations we will consider a partition, on some given time interval [0 ]T, and write (2.9) This partition is strict, 0 1t t …< < , and regular, the 1t k kt t −∆ = − are identical for indices k . Applying the Euler-Maruyama scheme to (8), we get, (2.10) Here The Gaussian process v is an independently and identically distributed (0 1)N , . Our stochastic system now, under the measure P , has the form: (2.11) Write 3. State Estimation Filters The approach we take to compute our filters is the so-called reference probability method. This technique is widely used in Electrical Engineering, see Elliott et al. (1995) and more recently Aggoun and Elliott (2004). We define a probability measure †P on the measurable space ( )FΩ, , such that, under †P , the following two conditions hold. 1. The state processes Z and are Markov chains initial distributions 0p and 0p respectively. 2. The observation process X , is independently and identically distributed and is Gaussian with zero mean and unit variance. With †P defined, we construct P , such that under P the following hold: FILTERING AND M-ARY DETECTION OF MARKOV 91 3. The state processes Z and are again Markov chains with initial distributions 0p and 0p respectively. 4. The sequence v, where (3.1) is a sequence of independently and identically distributed Gaussian (0 1)N , random variables. Write . Definition 1. For 1 2 …= , , , (3.2) (3.3) The "real world" probability P , is now defined in terms of the probability measure †P by setting † tG k dP dP | = Λ . Lemma 1. Under P , the sequence v, is a sequence of independently and identically distributed (0 1)N , random variables, where . That is, under P , (3.4) Lemma 2. Under the measure P , the process Z remains a Markov process, with transition matrix Π and initial distribution 0p . The proofs of Lemma 1 and 2 are routine. Remark 1. The objective in estimation via reference probability is to choose a measure †P which facilitates and/or simplifies calculations. In Filtering and Prediction, we wish to evaluate conditional expectations. Under the measure †P , our dynamics have the form: (3.5) In what follows we shall use the following version of Bayes' rule. LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 92 (3.6) Note that The following result is proven in Malcolm et al. (2008). Theorem 1. Information State Recursion. Suppose the Markov chains Z and are observed though the unit- delay discrete-time dynamics at (2.10). The information state for the corresponding filtering problem is computed by the recursion: (3.7) Here , (3.8) and (3.9) The recursion given in Theorem 1, provides a scheme to estimate the conditional probabilities for events of the form , given the information up to time k+1. In practice, one would use the vector-valued information state , to compute an estimate for the state . In general two approaches are adopted; one computes either a conditional mean, that is (3.10) or the so-called Maximum-a-Posteriori (MAP) estimate, that is (3.11) Marginal distributions for the Markov chains are obtained by multiplying on the right with the n -dimensional row vector (1 1)…, , or on the left with the m -dimensional column vector (1 1)…, , respectively. 4. M-ary Detection Filters FILTERING AND M-ARY DETECTION OF MARKOV 93 To denote a specific model hypothesis for the discrete-time dynamics given at (2.5), (2.7) and (2.10) we write, (4.1) Here . Using the simple random variable , as before, we are interested to compute the detector expectation (4.2) Here the sigma algebra is taken as generated by a model with parameter set , and similarly the Radon- Nikodym derivative , is constructed according to . Further, to make a clear distinction between the filter information state defined for specific model , and the corresponding un-normalised detector probability for model , we write, respectively Theorem 2 (M-ary Detection Filter) The M-ary detection filter for the model hypothesis is computed by the recursion Proof: LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 94 The expectation in the last line of the calculation is The normalized probabilities are computed by the normalized one step predictor information state, that is, for the model hypothesis and the event , we compute Here is the information state for the filter computed earlier. Since we need the normalised form of the expectation at (4.2), the -ary detector has the form: 5. Continuous-Time Dynamics We consider here a continuous time Markov chain . Again we use the canonical representation of an arbitrary Markov chain. That is, without loss of generality we take the state space for to be the set , whose elements are column vectors with unity in the position and zero elsewhere. The key benefit of this representation is that it admits the dynamics: FILTERING AND M-ARY DETECTION OF MARKOV 95 Here is a -martingale and is a time invariant rate matrix, whose elements are the infinitesimal intensities of . To denote an element of the matrix at row and column , we write . Here denotes an inner product. Now we consider the continuous-time dynamics (5.1) Under the state and observation process dynamics have the form: Let where is given by equation (5.1). Then the ‘real world’ probability is defined via Under the dynamics have the form: Notation: Suppose is any -adapted process and we wish to estimate . Using Bayes’ rule (Elliott et al. 1995) 6. Continuous-Time Detection Schemes State Estimation Filters With LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 96 Then . -ary Detection Filters The process Z takes values on a canonical basis of matrix-valued indicator functions, each of which jointly indicates a particular model hypothesis, and a particular value taken by the state process. FILTERING AND M-ARY DETECTION OF MARKOV 97 The corresponding normalized detection probabilities are computed, for example, by Write (6.1) define The process , defined by equation ( 6.1) satisfies the dynamics The symbol in the previous equation denotes a point-wise matrix product, where for two matrices of the same dimensions, the point-wise product is LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 98 Write Recalling the numerator in Bayes’ rule, we note that So, by computing the numerator in Bayes’ rule, we can readily compute the normalizing denominator . The matrix quantity , defined at (6.1), is an un-normalized conditional expectation, so, the corresponding normalized conditional expectation is computed by To recover the normalized -ary detection probabilities from the quantity , one computes The corresponding normalized detection probabilities are computed, for example, by Write (6.1) define The process , defined by equation ( 6.1) satisfies the dynamics FILTERING AND M-ARY DETECTION OF MARKOV 99 The symbol in the previous equation denotes a point-wise matrix product, where for two matrices of the same dimensions, the point -ise product i Write Recalling the numerator in Bayes’ rule, we note that So, by computing the numerator in Bayes’ rule, we can readily compute the normalising denominator . The matrix quantity , defined at (6.1), is an un-normalized conditional expectation, so, the corresponding normalized conditional expectation is computed by To recover the normalized -ary detection probabilities from the quantity , one computes 6. References AGGOUN, L. and ELLIOTT, R.J. 2004 Measure Theory and Filtering: Introduction with Applications. Cambridge Series in Statistical and Probabilistic Mathematics. BAUM, L.E. and PETRIE, T. 1966 Statistical Inference for Probabilistic Functions of Finite State Markov Chains. Annals of the Institute of Statistical Mathematics, 37: 1554-1563. LAKHDAR AGGOUN, MOHAMED AL-LAWATI and W.P. MALCOLM 100 DEMP, A.P., LAIRD, N.M. and RUBIN, D.B. 1977 Maximum Likelihood from Incomplete Data via the EM algorithm. Jour. of the Royal Statistical Society B 39: 1-38. ELLIOTT, C., ELLIOTT, R.J. and MALCOLM, W.P. 2005 Commodity prices and regime switching bases, Conference on Stochastic Modelling of Complex Systems, Daydream Island, Queensland Australia 10-16 July. ELLIOTT, R.J., AGGOUN, L. and MOORE, J.B. 1995 Hidden Markov Models: Estimation and Control. Springer-Verlag, New-York. Applications of Mathematics No. 29. ELLIOTT, R.J. 1982 Stochastic Calculus and Applications. Springer-Verlag, New-York. Applications of Mathematics No. 18. ELLIOTT, R.J., FISCHER, P. and PLATEN, E. 1999 Filtering and parameter estimation for a mean-reverting interest rate model, Canadian Applied Mathematics Quarterly, 7: 381-400. ELLIOTT, R.J. and KOPP, P.E. 2005 Mathematics of Financial Markets, 2nd Edition, Springer-Verlag. KLOEDEN, P.E. and PLATEN, E. 1992. Numerical Solution of Stochastic Differential Equations Springer- Verlag, New-York. Applications of Mathematics No. 23. MALCOLM, W.P., AGGOUN, L., and Al-LAWATIA, M. 2008. On A Markov Modulated Mean Reverting Finance Model. Journal of Science and Technology, Sultan Qaboos University. 13: 55-62 . POOR, H. 1988. An Introduction to Signal Detection and Estimation, Springer-Verlag, New York. Received 28 April 2009 Accepted 8 December 2009