Available online at www.HighTechJournal.org HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 15 ISSN: 2723-9535 Eye Tracking Algorithm Based on Multi Model Kalman Filter S. H. Ziafati Bagherzadeh 1*, S. Toosizadeh 1 1 Department of Electrical Engineering, School of Engineering, Islamic Azad University, Mashhad Branch, Mashhad, Iran. Received 14 November 2021; Revised 12 January 2022; Accepted 23 January 2022; Published 01 March 2022 Abstract One of the most important pieces of Human Machine Interface (HMI) equipment is an eye tracking system that is used for many different applications. This paper aims to present an algorithm in order to improve the efficiency of eye tracking in the image by means of a multi-model Kalman filter. In the classical Kalman filter, one model is used for estimation of the object, but in the multi-model Kalman filter, several models are used for estimating the object. The important features of the multiple-model Kalman filter are improving the efficiency and reducing its estimating errors relative to the classical Kalman filter. The proposed algorithm consists of two parts. The first step is recognizing the initial position of the eye, and Support Vector Machine (SVM) has been used in this part. In the second part, the position of the eye is predicted in the next frame by using a multi-model Kalman filter, which applies constant speed and acceleration models based on the normal human eye. Keywords: Eye Tracking; Multi-Model Kalman Filter; Support Vector Machine; Image Processing. 1. Introduction Eyes and their movements are very important in the detection of disease, needs, and emotions. In other words, by considering the geometry of the eye, its movement, and state, we can figure out the needs and emotions. The necessity of studying and developing eye tracking has special importance amongst researchers of different fields. When it comes to an accident, the eye’s first feature is being less vulnerable in comparison to other organs. Currently, researchers have presented various methods for eye tracking that can be used for developing useful and accurate eye tracking systems. Eye detection and tracking are used by researchers for face detection, expressing one’s emotions, and face recognition. Eye tracking methods can be divided into two categories as follows:  Passive methods based on images and active methods based on infrared light;  Classic passive methods detect faces based on eye shape and light intensity distribution. In infrared based methods, eye detection and tracking are done by taking the eye’s (pupil’s) reflection of infrared rays into measure. Classic methods in face detection can be divided into three categories including:  Pattern-based methods,  Appearance-based methods, and  Specifications-based methods. * Corresponding author: hassan.ziafati@mshdiau.ac.ir http://dx.doi.org/10.28991/HIJ-2022-03-01-02  This is an open access article under the CC-BY license (https://creativecommons.org/licenses/by/4.0/). © Authors retain all copyrights. https://creativecommons.org/licenses/by/4.0/ HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 16 In the pattern-based method, based on its shape, a general model of the eye is chosen, and then the image is scanned to find the eye by pattern research algorithms. Xiong et al. [1] suggested a method based on Houph Transformation for accurate face detection: Eye tracking by using the distance between the eyes Deformable patterns are widely used for eye detection [2, 3]. In these methods, a model of the eye would be designed to match the eye’s figure, which includes desirable features such as translatability, rotatability, and deformability; then, the eye would be detected by minimizing the energy. Appearance-based methods detect the eye by photometric appearance characteristics. These methods usually require a wide range of training data, eye descriptions in different positions, different facial expressions, and light conditions. Data in appearance-based methods is used to train neural networks or Support Vector Machine (SVM). Obviously, detection is based on categorization in these methods. Shah et al. (2013) [4] have used special face techniques for eye detection. In Huang et al. (2000) [5], the eye’s feature space is used for eye detection. This method benefits from desirable speed but lacks accuracy. In Huang et al. (1999) [6], image wavelet transformation and eye detection have been used with a recurrent neural network. Today, researchers use other neural network-based methods for face detection. For example, in [7], there are some proposed methods for improving eye detection by neural networks. The main weakness of appearance-based methods is the need for a large dataset in the training phase for face detection. Specification-based methods use some features like edges, color distribution, and so on for eye detection. A specifications-based method is used by Wang et al. (2016) [8]. For Vrânceanu et al. (2015) [9], and Dong et al. (2015) [10], specifications of the corners of eyes are used for face detection. In these references, images have been used for face detection. The main weaknesses of this method are inefficiency in the presence of the hair in the front part of the eye and face direction. In Morcego et al. (2016) [11], a method is proposed for estimating the parameters of the eye and tracking it. In this method, the eye model should be designed first. The corner of the eye is traced by the Kalman algorithm. The main weakness of this method is the need for a high-quality image for eye tracking. In summary, the classic detection and tracking methods of detecting and tracking by searching for form, appearance, and eye specifications are still applicable. In Lescroart et al. (2016) [12], and Amiel et al. (2015) [13], a wavelet filter has been used to reduce the transparency effect for eye detection. However, this filter lacks suitable efficiency for all transparencies. Therefore, classic eye detection methods have some challenges, such as face direction and the transparency of the image. In Zhu et al. (2007) [14], two new methods have been employed for face tracking based on the eye model. The first method detects the stared eye as 3D. Cornea is considered as a convex mirror in this method. Based on the features of an assumed convex mirror, the eye direction is obtained. This method is very complicated and is not feasible in practical use. In Yoo et al. (2005) [15], a method for eye tracking is presented based on pupillary light direction. Face tracking using the Kalman filter is another method for detecting eyes. In Zhu et al. (2002) [16], a combination of Kalman filter and data classification is used for eye tracking. This is a very suitable method for face tracking. A neutral Kalman filter has been used for face detection by Zhang (2010) [17]. This paper is set to be used for detecting and estimating one’s eye’s position in frames of a film by using a multi-model Kalman filter algorithm. In the presented method, fixed velocity and fixed acceleration models of the multi-model Kalman filter are employed due to the dynamics of the target (eye) [18-20]. In the second section, pupillary tracking based on the Kalman filter and its equations is explained. In the next section, the simulation method and its outcome are argued. In the next section, the simulation and its results are presented. The final part of this paper provides the results and insights for more research in the future. 2. Materials and Methods 2.1. Eye Tracking by the use of Kalman Filter Kalman filter is a set of recursive equations that are used to estimate an object’s position and the degree of uncertainty in the next frame. In the Kalman filter, estimation of the next position is done by measuring data in previous frames, and then, in the next step, the previous position vector is estimated by the data of the current frame. It should be noted that the Kalman filter has a minimum Euclidean norm for estimating the object position with linear dynamics. Then, the pupil tracking is explained by the Kalman filter algorithm. In this section, consider the dynamic of eye movement as Equation 1. In this equation, 𝑥𝑘 is the mode vector of the system dynamic in frame K, 𝑦𝑘 is the output vector in frame K, 𝑤𝑘 is the process noise in frame K and 𝑣𝑘 is measurement noise in frame K. In the Kalman filter, the process and measurement noises are considered white noise. Covariance matrices of process and measurement of noises are obtained based on 𝑤𝑘 and 𝑣𝑘 data. In this section, it is assumed that the signals of 𝑤𝑘 and 𝑣𝑘 are independent and their covariance matrix is Q and R. Also, system’s dynamic mode vector is (2). In this equation, [𝑐𝑘 𝑟𝑘] is eye pixel position in k frame and [�̇�𝑘 �̇�𝑘] is eye speed in frame K. It is necessary to note that in Kalman filter, the entry is [𝑐𝑘 𝑟𝑘] and the output is [𝑐𝑘 𝑟𝑘 �̇�𝑘 𝑟�̇�] vector estimation. The matrix can be written based on the data. HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 17 { 𝑥𝑘+1 = 𝐴𝑥𝑘 + 𝑤𝑘 𝑦𝑘 = 𝐶𝑥𝑘 + 𝑣𝑘 (1) 𝑥(𝑡) = [𝑐𝑘 𝑟𝑘 �̇�𝑘 𝑟�̇�] (2) 𝐶 = [ 1 0 0 0 0 1 0 0 ] (3) Let’s take �̂�𝑘|𝑘 as posterior estimate 𝑥𝑘 in K frame, �̂�𝑘|𝑘−1 as prior estimate 𝑥𝑘 in frame K, 𝑃𝑘|𝑘 as posterior covariance matrix in K frame, 𝑃𝑘|𝑘−1 as prior covariance matrix in K frame and 𝐾𝑘 as the gain of Kalman filter in K frame. In this case, updating Kalman filter can be done in the K frame based on the Equations 4, and 5: �̂�𝑘|𝑘−1 = 𝐴�̂�𝑘−1|𝑘−1 𝑃𝑘|𝑘−1 = 𝐴𝑃𝑘−1|𝑘−1𝐴 𝑇 + 𝑄 𝑆𝑘 = 𝐶𝑃𝑘|𝑘−1𝐶 𝑇 + 𝑅 (4) 𝐾𝑘 = 𝑃𝑘|𝑘−1𝐻 𝑇𝑆𝑘 −1 �̂�𝑘|𝑘 = �̂�𝑘|𝑘−1 + 𝐾𝑘(𝑦𝑘 − 𝐶�̂�𝑘|𝑘−1) 𝑃𝑘|𝑘 = (𝐼 − 𝐾𝑘𝐻𝑘)𝑃𝑘|𝑘−1 (5) The purpose of this section is to explain the proposed method based on multi-model algorithm for optimizing the estimation of eye position. Therefore, the block diagram of the proposed method has been studied for eye tracking in a film. Third section deals with defect detection method in a frame. Support vector machine has been used in this method. Forth section explains multi-model Kalman filter algorithm and how to use it in eye tracking. Eye Detection Input Image No Multi-Model Kalman Filter Based Eye track No Update probability of each model Yes Yes State Estimation Figure 1. Block diagram of the proposed method of eye tracking Diagram block of eye tracking algorithm is shown in Figure 1. According to this figure, in the first step, the algorithm of eye tracking can detect the first position of eye in the first frame. Clearly, support vector machine algorithm has been used in this method. After detecting the eye, pixel position data from each eye is entered to multi- model Kalman filter. The purpose of this method is to predict the eye position in the next frame. After solving the equation of filter, estimation of eye position, pixel position in the next fame, proposed algorithm consider the accuracy of estimation based on eye data in frame. In proposed algorithm, the distance between the actual amount of eye position and the estimated amount is used for evaluation. If the estimated accuracy is appropriate, updated probability of each model and estimation of eye position is done next. Also in the case of higher uncertainty the proposed algorithm returns to eye tracking. Understandably, multi-model Kalman filter is useful not only for improving the performance of eye tracking, but also for optimizing estimation errors of eye tracking and the existence of noise in measured data. The filter nature and Gaussian white noise in measured data are the reasons of this. The first step of eye tracking is detecting it. In this paper, two-level algorithm is used for detection of first position of eye. By two-level algorithm we mean the use of two steps for estimating eye position in a single frame of a film. It should be noted that said algorithm has almost a lot of volume of calculation and cannot be used in every frame. Each level is explained in the following. General schematic method of eye tracking is shown in Figure 2. HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 18 Face Detection Eye Detection No Input Eye Position Figure 2. Used eye tracking algorithm First level pursues identify the face in a frame. In other words, the pixel position of the face in an image is the output of first level. As mentioned before, the support vector machine is used for doing this work. For evaluating used algorithm in the first level, there is an image like a). The result of face detection (face level) is shown in b). It is clear that the proposed algorithm can identify the face in the studied image well. In the first level, face position in the image is clear. The purpose of second level is to analyze the data of pixels which are in the face position for the face detection. Also support vector machine is used for this level. The result of second level is shown in c). In this image, the left eye has the horizontal pixel position 466 and vertical 265 and right eye has the horizontal pixel position 765 and vertical pixel 276. It should be noted that in those data, eye pixel position is used as multi-model Kalman filter entrance. 2.2. Eye Tracking Algorithm In this study, multi-model Kalman filter is used for estimating eye position in the next frame of film. Therefore, multi-model Kalman filter and its use ought to be explained. Multi-model Kalman filter estimator is a sub-optimal hybrid filter and it is very useful for hybrid estimation of target’s position. The main strength of this method is the estimation of system’s modes with several behavioral models and the ability to switch between them. In practice, we can take multi-model Kalman filter as a self-tuning filter with flexible bandwidth. Assume that the equation of state space of moving object is Equation 6.      ( 1) , ( 1) ( ) , ( 1) , ( 1) ( ) ( ) ( ) ( ) x k F k m k x k G k m k v k m k z k H k x k w k         (6) In the Equation 6, 𝑥 𝜖 ℜ𝑛𝑥 is object mode vector, 𝑧 𝜖 ℜ𝑛𝑧 is measured output vector, 𝑤 𝜖 ℜ𝑛𝑧 and 𝑤 𝜖 ℜ𝑛𝑢 are independent white Gaussian noise vectors, and𝑄𝑣 and 𝑅𝑤 are covariance. Also 𝑚(𝑘) shows the mode of system, 𝐹 represents system dynamic matrix and H represents the measurement matrix. Generally, various models have to be used to describe the system since there is no exact system model. In this method, we show the probability of mode I and K being used at the same as following: 𝑀𝑖(𝑘) = {𝑚(𝑘) = 𝑚𝑖}. In the presented method, it is assumed that order of system’s model changes follow a Markov Chain with the transition probability of Equation 7;  ( 1) | ( ) ( )j i ijP m k m k P k  (7) (a) (b) (c) Figure 3. The process of personal algorithm of eye tracking: (a) Man image; (b) Recognition range of face; (c)The results of eye detection algorithm HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 19 Which, “∑ 𝑃𝑖𝑗(𝑘)𝑟 𝑗=1 = 1, 𝑖 = 1, 2, … . . , 𝑟” in above equation is correct based on Markov Chain Mathematics. Also in this method, a Markov transition matrix is used for describing the probability of the object in explained model. The probability of models is updated at the end of each algorithm repetition, and obtained results such are considered as weighting coefficient of each model. In short, a full cycle of implementation of the multi- model Kalman filter algorithm is as follows: First Step: Mixing Probabilities The probability of 𝑀𝑖 model affecting the 𝑘 − 1 time is calculated based on the probability of 𝑀𝑗 model affecting 𝐾 time and is shown in this Equations 8 and 9. ( ( ) | ( )) ( )i iP M k Z k k (8)    -1 | -1| -1 ( -1) | ( ), 1 ( -1) k i j i j ij i j k k P M k M k Z p k c     (9) In Equation 9, the estimated probability for I-th mode is c- j which is calculated as in Equation 10. 1 ( -1) r j ij i i c p k   (10) Second Step: Calculating Initial Mixing Values According to previous estimations, for state variable and covariance matrix (in order 𝑥 �̂� (𝑘 − 1|𝑘 − 1) and 𝑥 �̂� = 𝑃𝑖(𝑘 − 1|𝑘 − 1) primal combination conditions are calculated as in Equation 11.              | 1 | 1 -1 | -1 -1 | -1 -1 | -1 -1 | -1 -1 | -1 -1 | -1 ( -1 | -1) - ( -1 | -1) ( -1 | -1) - ( -1 | -1) r oj i i j i r oj i i j i T i oj i oj x k k x k k k k P k k k k P k k x k k x k k x k k x k k                 (11) Third step: Calculation of Fitness Function for each Model First, by use of Kalman filter Equation 12         , -1 | -1 ( -1) ( -1| -1) ( -1) ( -1) | | -1 ( ) ( ) ( ) ( ) - ( ) | -1 ( | -1) ( -1) ( -1| -1) ( -1) ( ) ( ) ( ) ( | -1) ( ) ( ) ( ) ( | -1) ( ) ( ) ( i i i i i f i i i i i T T i i i i T i i i i T i i i i i x k k F k x k k G k v k x k k x k k K k v k v k z k H k x k k P k k F k P k k F k GQ k G S k H k P k k H k R k K k P k k H k S k P k           | ) ( | -1) - ( ) ( ) ( )T i i i ik P k k K k S k K k (12) Then, fitness function will be achieved for each model Equation 13; 1 - -12( ) ( );0; ( ) 2 ( ) exp -0.5 ( ) ( ) ( )T j j j j j j jk N v k S k S k v k S k v k          (13) Forth Step: Model Probability According to Equations 14 and 15, the probability of each model will be calculated. 1 ( ) r j j j c k c    (14) 1 ( ) ( )j j jk k c c    (15) HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 20 Fifth Step: Estimation of Modes and Calculating the Covariance Matrix 1 ( | ) ( | ) ( ) r i i i x k k x k k k   (16) 1 ( | ) ( | ) - ( | ) ( | ) ( ) ( | ) - ( | ) i i r i T i i P k k x k k x k k P k k k x k k x k k                  (17) The steps stated above, are for implementing the next step after measuring by sensor and by each sensor measurement, the steps are repeated. In Figure 4, general diagram block of multi-model Kalman filter method is shown. Figure 4. General block diagrams of multi model Kalman filter method As was stated before, several models are used in this method. The models are divided into two main categories: Constant Speed Model and Constant Rotation Rate. In the constant speed model, there is fixed speed object and in constant rotation rate there is constant angular speed. Equation 18 elaborate the argument.   1 2 2 2 2 sin 1- cos 1 0 - 1 0 cos 0 -sin , 0 1 1- cos sin 0 1 0 sin 0 cos : 0, ( , ) . t t t CV CT t x y x Fx Gu T T T T T T F diag G diag FT T T T T T Where u N diag                                                     (18) In this equation, Δ𝑇 is the time of sampling and 𝜔 is the angular speed vector of object. Needless to say, all the models are linear. 3. Result and Discussion The purpose of this study is to present eye tracking algorithm based on multi-model filter. Based on the previous arguments, there are several models instead of one model which are used in Kalman filter method. It is clear that the important weakness of Kalman filter is constant eye dynamic movement. In other words, the very reason of employing several models instead on one is optimizing the efficiency of proposed algorithm when it faces dynamic objects. In this section, we are to simulate multi-model Kalman filter while taking different models into account. It is crucial to say that in this simulation, two models of Constant Speed Model and Constant Rotation Rate are considered. { �̅�𝑘+1 = 𝐴𝑖�̅�𝑘 + 𝐵𝑖𝑣𝑘 𝑦𝑘 = 𝐶𝑖�̅�𝑘 + 𝑘 �̅�𝑘 = [𝑥𝑘�̇�𝑘𝑦𝑘�̇�𝑘] 𝑇 𝒜1 = [ A1 0 0 A1 ] , A1 = [ 1 T 0 1 ] , ℬ1 = [ B1 0 0 B1 ] Object dynamic (19) HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 21 𝐵1 = [ 𝑇2 2⁄ 𝑇 ] Constant speed matrix model (20) �̅�𝑘 = [𝑥𝑘�̇�𝑘�̈�𝑘𝑦𝑘�̇�𝑘�̈�𝑘] 𝑇 𝒜2 = [ 𝐴2 0 0 𝐴2 ], 𝐴1 = [ 1 𝑇 𝑇2 2⁄ 0 1 𝑇 0 0 1 ] , ℬ2 = [ 𝐵2 0 0 𝐵2 ] , 𝐵2 = [ 𝑇2 2⁄ 𝑇 1 ] Constant acceleration (21) The results of the algorithm simulation are explained in the following and are shown in Figure 5. This model follows the object very well. Figure 5. Results of multi-model interference filter simulation Simulation The purpose of this section is to consider the proficiency of proposed algorithm for eye tracking in an image in different frames. As explained before, there are different models for eye tracking such as constant speed, constant acceleration, constant rotation, and constant mass. Their efficiencies are different based on the type of object changes. Since we set to track eye, constant speed and constant acceleration models are used in this paper. Considering previous data and also covariance matrix of noise measurement, for constant speed Equation 22 and for constant acceleration Equation 23; 𝑅 = [ 4 0 0 4 ] ; 𝑄 = [ 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ] (22) 𝑅 = [ 4 0 0 4 ] ; 𝑄 = [ 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0.5 0 0 0 0 0 0 0.5] (23) In the following part, we consider the efficiency of proposed algorithm in different frames of a film for eye tracking. First Scenario: The purpose is to trace the pupil by constant face. The face is fixed and the pupils have an anti-clockwise circular motion. Different frames and the results are shown in Figure 6. It is clear that the proposed algorithm could trace the pupil well. -4 -2 0 2 4 6 8 10 12 14 16 -30 -25 -20 -15 -10 -5 0 5 Estimates produced by IMM-filter. Measurement True trajectory Filtered HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 22 (a) (b) (c) (d) (e) Figure 6. The results of eye tracking in the first scenario: a) frame 1; b) frame 50; c) frame 100; d) frame 150; c) frame 200 In Figures 7 and 8, the errors of estimating eye position to the real mode by proposed algorithm and Kalman filter are shown. It should be noted that in this figure, measurement of the horizontal axis is pixel. Figure 7. The error of left eye estimation of position 0 50 100 150 200 250 -4 -3 -2 -1 0 1 2 3 4 frame number E rr [L e ft E y e ] X P o s it io n IMM Kalman 0 50 100 150 200 250 -3 -2 -1 0 1 2 3 frame number E rr [L e ft E y e ] Y P o s it io n IMM Kalman HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 23 Figure 8. The error of right eye estimation of position Finally, in Table 1 compares results of noise measurement on eye position to Kalman filter. Table 1. Consideration of the efficiency of Kalman filter in first scenario Kalman Filter Proposed Filter Measurement Noise Right Left Right Left 62.184 65.26 8.743 6.62 %80 Nominal value 93.66 67.57 12.53 8.02 %120 Nominal value 111.88 95.32 16.537 24.18 %200 Nominal value 148.48 149.10 19.26 39.54 %300 Nominal value Second Scenario: The purpose is eye tracking by animated face. At first, face moves to right; and then left. The results of this simulation are shown in different frames. It is clear that the proposed model could trace the pupil very accurately. (a) (b) (c) (d) (e) Figure 9. The results of eye tracking in the second scenario: a) frame 1; b) frame 50; c) frame 100; d) frame 150; c) frame 200 0 50 100 150 200 250 -4 -2 0 2 4 frame number E rr [R ig h t E y e ] X P o s it io n IMM Kalman 0 50 100 150 200 250 -3 -2 -1 0 1 2 3 frame number E rr [R ig ht E ye ] Y P os iti on IMM Kalman HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 24 Figures 10 and 11 show how off the trace has been compared to real state of the eye. Figure 10. The error of left eye estimation of position Figure 11. The error of right eye estimation of position Finally, the results of the different noise measurement on eye position are shown in Table 2. Clearly enough, proposed method works a lot better than Kalman filter. Table 2. Consideration of proposed filter in the second scenario Kalman Filter Proposed Filter Measurement Noise Right Left Right Left 79.5 59.7 8.24 8.5 %80 Nominal value 83.35 63.3 8.85 8.43 %120 Nominal value 102.59 95.16 13.55 22.93 %200 Nominal value 150.45 135.69 26.22 49.45 %300 Nominal value Third Scenario: The purpose is eye tracking by animated camera. The position of face is fixed but the camera is moving. The results are shown in and this model has high efficiency. Results are shown in Figure 12. 0 50 100 150 200 250 -2 -1 0 1 2 frame number E rr [L e ft E y e ] X P o s it io n IMM Kalman 0 50 100 150 200 250 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 frame number E rr [L e ft E y e ] Y P o s it io n IMM Kalman 0 50 100 150 200 250 -3 -2 -1 0 1 2 3 frame number E rr [R ig h t E y e ] X P o s it io n IMM Kalman 0 50 100 150 200 250 -3 -2 -1 0 1 2 3 4 frame number E rr [R ig h t E y e ] Y P o s it io n IMM Kalman HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 25 (a) (b) (c) (d) (e) Figure 12. The results of eye tracking in the first scenario: a) frame 1; b) frame 50; c) frame 100; d) frame 150; c) frame 200 Figures 13 and 14 show how off the trace has been compared to real state of the eye. Figure 13. The error of left eye estimation of position 0 50 100 150 200 250 -3 -2 -1 0 1 2 3 frame number E rr [L e ft E y e ] X P o s it io n IMM Kalman 0 50 100 150 200 250 -4 -3 -2 -1 0 1 2 3 frame number E rr [L e ft E y e ] Y P o s it io n IMM Kalman HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 26 Figure 14. The error of right eye estimation of position At the end, the results of the different noise measurements on eye position are provided in Table 3. The efficiency of the proposed method is considerably high. Table 3. Consideration of proposed filter in the third scenario Kalman Filter Proposed Filter Measurement Noise Right Left Right Left 44.24 64.96 8.81 9.2325 %80 Nominal value 72.58 73.86 9.65 10.72 %120 Nominal value 79.35 86.29 11.21 17.53 %200 Nominal value 104.52 94.32 20.31 25.97 %300 Nominal value 4. Conclusion In this study, the results of the proposed algorithm simulation for eye tracking in different frames were studied. In the Kalman filter, there is just one model, but in the multi-model Kalman filter, there are different models for estimating the object, of which two models, constant speed and constant acceleration, are used in this paper. Furthermore, a support vector machine is also used in this study for identifying the eye. It is clear that the proposed algorithm could trace the eye’s position in the film with high accuracy. There are some important notes for readers to discuss the proposed models: Multi-model Kalman has better efficiency than the Kalman filter; the proposed model does not need an accurate model of object dynamics (eye); it can be used in real time. 5. Declarations 5.1. Author Contributions S.H.Z.B. and S.T. contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript. All authors have read and agreed to the published version of the manuscript. 5.2. Data Availability Statement The data presented in this study are available on request from the corresponding author. 5.3. Funding The authors received no financial support for the research, authorship, and/or publication of this article. 5.4. Informed Consent Statement Informed consent was obtained from all individual participants included in the study. 5.5. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 0 50 100 150 200 250 -4 -3 -2 -1 0 1 2 frame number E rr [R ig h t E y e ] X P o s it io n IMM Kalman 0 50 100 150 200 250 -3 -2 -1 0 1 2 3 frame number E rr [R ig h t E y e ] Y P o s it io n IMM Kalman HighTech and Innovation Journal Vol. 3, No. 1, March, 2022 27 6. References [1] Xiong, J., Xu, W., Liao, W., Wang, Q., Liu, J., & Liang, Q. (2013). Eye control system based on ameliorated though transform algorithm. IEEE Sensors Journal, 13(9), 3421–3429. doi:10.1109/JSEN.2013.2262934. [2] Yuille, A. L., Hallinan, P. W., & Cohen, D. S. (1992). Feature extraction from faces using deformable templates. International Journal of Computer Vision, 8(2), 99–111. doi:10.1007/BF00127169. [3] Xie, X., Sudhakar, R., & Zhuang, H. (1994). On improving eye feature extraction using deformable templates. Pattern Recognition, 27(6), 791–799. doi:10.1016/0031-3203(94)90164-3. [4] Shah, J., Sharif, M., Raza, M., & Azeem, A. (2013). A survey: Linear and nonlinear PCA based face recognition techniques. International Arab Journal of Information Technology, 10(6), 536–545. [5] Huang, W., & Mariani, R. (2000). Face detection and precise eyes location. Proceedings - International Conference on Pattern Recognition, 15(4), 722–727. doi:10.1109/icpr.2000.903019. [6] Huang, J., & Wechsler, H. (1999). Eye detection using optimal wavelet packets and radial basis functions (RBFs). International Journal of Pattern Recognition and Artificial Intelligence, 13(7), 1009–1025. doi:10.1142/S0218001499000562. [7] Reinders, M. J. T., Koch, R. W. C., & Gerbrands, J. J. (1996). Locating facial features in image sequences using neural networks. Proceedings of the International Conference on Automatic Face and Gesture Recognition, 230–235. doi:10.1109/afgr.1996.557269. [8] Wang, C., Shi, F., Xia, S., & Chai, J. (2016). Real-time 3D eye gaze animation using a single RGB camera. ACM Transactions on Graphics, 35(4), 1–14. doi:10.1145/2897824.2925947. [9] Vrânceanu, R., Florea, C., Florea, L., & Vertan, C. (2015). Gaze direction estimation by component separation for recognition of Eye Accessing Cues. Machine Vision and Applications, 26(2–3), 267–278. doi:10.1007/s00138-014-0656-8. [10] Dong, C., Wang, X., Pei-Hua, C., & Pu-Liang, Y. (2015). Eye Detection Based on Integral Projection and Hough Round Transform. Proceedings - 2015 IEEE 5th International Conference on Big Data and Cloud Computing, BDCloud 2015, 252– 255. doi:10.1109/BDCloud.2015.34. [11] Morcego, B., Argilés, M., Cabrerizo, M., Cardona, G., Pérez, R., Pérez-Cabré, E., & Gispets, J. (2016). Blinking supervision in a working environment. Journal of Biomedical Optics, 21(2), 025005. doi:10.1117/1.jbo.21.2.025005. [12] Lescroart, M. D., Kanwisher, N., & Golomb, J. D. (2016). No evidence for automatic remapping of stimulus features or location found with fMRI. Frontiers in Systems Neuroscience, 10(Jun), 53. doi:10.3389/fnsys.2016.00053. [13] Amiel, F., Barry, B., Trocan, M., & Swynghedauw, M. (2015). Real time image compression for eye tracking applications. 2015 IEEE 6th Latin American Symposium on Circuits and Systems, LASCAS 2015 - Conference Proceedings, 1–4. doi:10.1109/LASCAS.2015.7250463. [14] Zhu, Z., & Ji, Q. (2007). Novel eye gaze tracking techniques under natural head movement. IEEE Transactions on Biomedical Engineering, 54(12), 2246–2260. doi:10.1109/TBME.2007.895750. [15] Yoo, D. H., & Chung, M. J. (2005). A novel non-intrusive eye gaze estimation using cross-ratio under large head motion. Computer Vision and Image Understanding, 98(1), 25–51. doi:10.1016/j.cviu.2004.07.011. [16] Zhu, Z., Ji, Q., Fujimura, K., & Lee, K. (2002). Combining Kalman filtering and mean shift for real time eye tracking under active IR illumination. In Proceedings - International Conference on Pattern Recognition, IEEE, 16(4), 318–321. doi:10.1109/icpr.2002.1047460. [17] Zhang, Z., & Zhang, J. (2010). A new real-time eye tracking based on nonlinear unscented Kalman filter for monitoring driver fatigue. Journal of Control Theory and Applications, 8(2), 181–188. doi:10.1007/s11768-010-8043-0. [18] Li, Y., Deng, J., Wu, Q., & Wang, Y. (2021). Eye-Tracking Signals Based Affective Classification Employing Deep Gradient Convolutional Neural Networks. International Journal of Interactive Multimedia and Artificial Intelligence, 7(2), 34. doi:10.9781/ijimai.2021.06.002. [19] Barbara, N., Camilleri, T. A., & Camilleri, K. P. (2020). EOG-Based Ocular and Gaze Angle Estimation Using an Extended Kalman Filter. ACM Symposium on Eye Tracking Research and Applications. doi:10.1145/3379156.3391357. [20] Wu, Q., Dey, N., Shi, F., Crespo, R. G., & Sherratt, R. S. (2021). Emotion classification on eye-tracking and electroencephalograph fused signals employing deep gradient neural networks. Applied Soft Computing, 110, 107752. doi:10.1016/j.asoc.2021.107752.