Microsoft Word - 294-2485-1-LE-rev2 ACTA IMEKO  ISSN: 2221‐870X  November 2016, Volume 5, Number 3, 24‐31    ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 24  Neural network approach to reduce dynamic measurement  errors  Andrei S. Volosnikov, Aleksandr L. Shestakov  South Ural State University, Lenin prospekt 76, 454080 Chelyabinsk, Russian Federation      Section: RESEARCH PAPER   Keywords: dynamic measurement errors; neural network model; inverse sensor model; recovery of sensor input signal; dynamic measurement data  processing  Citation: Andrei S. Volosnikov, Aleksandr L. Shestakov, Neural network approach to reduce dynamic measurement errors, Acta IMEKO, vol. 5, no. 3, article 5,  November 2016, identifier: IMEKO‐ACTA‐05 (2016)‐03‐05  Section Editor: Franco Pavese, Italy  Received October 7, 2015; In final form August 10, 2016; Published November 2016  Copyright: © 2016 IMEKO. This is an open‐access article distributed under the terms of the Creative Commons Attribution 3.0 License, which permits  unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited  Corresponding author: Andrei S. Volosnikov, e‐mail: volosnikovas@susu.ru    1. INTRODUCTION  The output of a measuring system may suffer from dynamic errors. An input signal recovery approach is presented that reduces these dynamic errors [1]. This problem is ill-posed and requires the numerical solution to the convolution integral equation [2], [3]. However, the analysis of measuring systems can be made in terms of the control theory [4], as well as of the theory of automatic control systems sensitivity [5], [6]. The automatic control theory approach effectively improves the dynamic measurement accuracy [7]-[9]. Along with it, the artificial neural network (ANN) approach to designing the dynamic models of measuring systems and algorithms for the data processing of dynamic measurements is a way of intelligent measuring systems development. Recent publications reflect researchers’ interest in the matter of dynamic measurement error reduction. In [10] the double channel correction method is proposed. This method is developed for improving the dynamic properties of methane sensors on the basis of two measurement transducers. The method assumes that they are described by identical first-order models with different time constants. Reference [11] deals with the dynamic compensation of a sensor approach based on an ANN infinite impulse response filter. This filter represents the inverse model of the sensor that recovers the sensor input signal. In the present paper the ANN inverse sensor model is also considered. However, this model is based on the ANN finite response filter that ensures stability of the inverse sensor model. Moreover, the proposed approach without reference to the order of the sensor model represents its inverse model as a special designed filter and a set of identical ANN sections that implement only the inverse model of the first-order system. Furthermore, the criterion of training sets formation and their length definition on the stage of the ANN model adjustment is proposed. In Section 2 of the paper the ANN representation of the direct model of a sensor is discussed. On the basis of this representation in Section 3 the ANN inverse model of the sensor in three different representations is proposed. In Section 4 results of experimental data processing are given in order to validate the efficiency of the proposed approach. Finally, in the concluding section the major results are summarized. 2. NEURAL NETWORK DIRECT MODEL OF A SENSOR  Suppose a primary measuring transducer (sensor) is described by the transfer function (TF) as follows: ABSTRACT  The neural network inverse model of a sensor with filtration of the sequentially recovered signal is considered. This model effectively  reduces the dynamic measurement errors due to deep mathematical processing of measurement data. The result of the experimental  data processing of a dynamic temperature measurement validates the efficiency of the proposed neural network approach to reduce  dynamic measurement errors.  ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 25                       n lj j l j jjj m ki i k i iii s pTpTpT pTpTpT K pU pY pW 1 1 1 11 22 1 1 2 1 22 22 2 0 112 112   , (1) where U and Y are the sensor input and output signals respectively; jT1 and iT2 are time constants; j1 and i2 are damping coefficients; mi ...,,2,1 , nj ...,,2,1 ( nm ); 0K is the static gain; p is the complex number frequency. The continuous TF(1) of the sensor can be represented in the following way:       0111 01 1 1 ... ... apapap bpbpbpb pU pY pW n n n m m m m s        , (2) where  022 ,, KTbb iiii  for mi ...,,2,1 are coefficients depending on the parameters of (1) numerator and the static gain;  jjjj Taa 11 , for nj ...,,2,1 are coefficients depending on the parameters of (1) denominator. The discrete analogue of the continuous TF(2) in general can be represented as follows:       nn n n s zzz zzz zU zY zW        ...1 ... 2 2 1 1 2 2 1 10 , (3) where )(zU and )(zY are the z-transformation of the sensor input and output signals, respectively;  Taabb nmii ,,...,,,..., 100   for ni ...,,1,0 and  Taabb nmjj ,,...,,,..., 100  for nj ...,,2,1 are coefficients depending on the parameters of (2) and the sampling period T . The difference equation corresponding to the discrete TF(3) of the sensor is as follows:         n j j n i i jkuikyky 01  , (4) where  ku and  ky are samples of the sensor input and output signals, respectively, at discrete times Tktk  for ...,2,1,0k . The relationship between the output and input of the discrete sensor model is described by the following recurrence equation derived from the previous one:         n j j n i i jkuikyky 01  . (5) The parameters of the discrete model (3) can be determined on the basis of the ANN direct sensor model shown in Figure 1. This model is the recurrent ANN [12] consisting of a single neuron with the linear activation function  netfa and the zero bias. The structure of this model is fully consistent with the recurrence equation (5). The recurrence equation that determines the relationship between input and output of the ANN direct sensor model is as follows:           n j j n i ia jkuwikyvnetnetfky 01 ** , (6) where  ku and  ky* are samples of the sensor input signal and the output signal of the ANN direct sensor model, respectively, at discrete times Tktk  for ...,2,1,0k ; jw for nj ...,,1,0 and iv for ni ...,,2,1 are adjustable coefficients (weights) of the ANN direct sensor model. By means of an appropriate procedure of the input and target training sets formation, that reflects the relationship between the input and output of the discrete sensor model (3), the weights of the ANN direct sensor model can be adjusted during the training process so that the samples of the ANN sensor model output will be equal to the respective samples of the continuous sensor model (1) output for a given level of the accuracy. The indicated possibility follows from the linearity and the conformity of the discrete and the ANN direct models of the sensor. Indeed, if    kyky * for ...,2,1,0k , then from (5) and (6) the following equality can be obtained:              n j j n i i n j j n i i jkuwikyv jkuiky 01 01  . (7) Provided the sensor input is nonzero, the last equality becomes the identity only when jj w for nj ...,,1,0 and ii v for ni ...,,2,1 . The functional block diagram of the ANN direct sensor model training is shown in Figure 2. The procedure of the ANN direct sensor model training (i.e. its weights adjustment) consists in minimization of the training error represented as the aggregate standard deviation for all 1N samples of the input training set )(khu between the target )(khy and actual )( * kh y outputs of the ANN direct sensor model: Figure 1. Block diagram of the ANN direct sensor model. ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 26         N k yy khkh N E 0 2*1 . (8) 3. NEURAL NETWORK INVERSE MODEL OF A SENSOR  The approach to the ANN direct sensor model creation discussed in the previous section can be used for the solution to the problem of the sensor (1) dynamic measurement error reduction. Then, this problem is formulated as the problem of the dynamically distorted sensor input signal recovery on the basis of the respective samples of its output signal. Considering the indicated formulation it is necessary on the basis of the ANN direct sensor model and the functional block diagram of its training to create the ANN inverse sensor model and, similarly, the functional block diagram of its training. This ANN inverse sensor model should provide the recovery of the dynamically distorted sensor input signal, i.e. implement the inverse relationship between the sensor input and output. 3.1. General representation  The discrete model (3) of the sensor is considered to obtain the structure of the ANN inverse sensor model. This TF is cast to the inverse form as follows:           n n n n nn nn n n n n n n n n s zzz zzz zzz zzz zzz zzz zzz zzz zY zU zU zY zW                                                            ...1 ... ...1 ... 1 ... ...1 ...1 ... 2 2 1 1 2 2 1 10 0 2 0 21 0 1 0 2 0 21 0 1 0 2 2 1 10 2 2 1 1 1 2 2 1 1 2 2 1 10 1 1 (9) where 0 0 1    , 0   ii  , 0   ii  for ni ...,,2,1 . The difference equation corresponding to the inverse discrete TF(9) of the sensor is as follows:         n j j n i i jkyikuku 01  , (10) where  ku and  ky are samples of the sensor input and output signals, respectively, at discrete times Tktk  for ...,2,1,0k . The relationship between the input and output of the inverse discrete sensor model is as the following recurrence equation derived from the previous one:         n j j n i i jkyikuku 01  . (11) The structure of (11) for the inverse discrete sensor model is the same as the structure of (5) for the direct discrete sensor model. Therefore the structure of the ANN inverse sensor model will also be the same as the structure of the ANN direct sensor model. The block diagram of the ANN inverse sensor model is shown in Figure 3. This model is the recurrent ANN consisting of a single neuron with the linear activation function  netfa and zero bias. The structure of this model is fully consistent with the recurrence equation (11). The recurrence equation that determines the relationship between the input and output of the ANN inverse sensor model is as follows:           n j j n i ia jkywikuvnetnetfku 01 ** , (12) where  ky and  ku* are samples of the sensor output signal and the output signal of the ANN inverse sensor model, respectively, at discrete times Tktk  for ...,2,1,0k ; jw for nj ...,,1,0 and iv for ni ...,,2,1 are weights of the ANN inverse sensor model. The criterion for the ANN inverse sensor model training as in the case of the ANN direct sensor model training is the minimum of the training error represented as the standard deviation between the target and actual outputs of the ANN inverse sensor model. Figure 2. Functional block diagram of the ANN direct sensor model training. ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 27  Obviously, both in the criterion and in the functional block diagram of the ANN inverse sensor model training it is necessary to swap the input and target training sets towards the criterion and the functional block diagram of the ANN direct sensor model training. This is the implementation of the inverse learning approach [13], applied to designing the dynamic control systems, in designing the dynamic measuring systems. 3.2. Cascade representation  In order to avoid possible problems with the stability of the inverse model discussed, the cascade representation of the ANN inverse sensor model in the form of first- and second- order sections is proposed. This approach is based on the representation of the direct sensor model (1) in the following form of r second-order and s first-order cascades with real coefficients:         s j j r i is pWpWpW 1 1 1 2 , (13) where the discrete TF of the second-order cascade in general is as follows   2 2 1 1 2 2 1 10 2 1      zz zz zW ii iii i   (14) and the discrete TF of the first-order cascade in general is as follows:   1 1 1 10 1 1      z z zW j jj j   . (15) Inverse TFs of cascades  pW i2 and  pW j1 derived analogically to (9) are as follows:   2 2 1 1 2 2 1 101 2 1       zz zz zW ii iii i   , (16)   1 1 1 101 1 1       z z zW j jj j   . (17) The functional block diagram of the ANN inverse model of the sensor in the cascade representation is shown in Figure 4 (where ][2 iC are sections implementing the inverse model (16) of second-order cascades  pW i2 for rj ...,,2,1 and ][1 jC are sections implementing the inverse model (17) of first-order cascades  pW j1 for si ...,,2,1 ). Thus, the structure of the cascade ANN inverse model of the sensor corresponds to the structure of the sensor in the cascade representation. Therefore, each section is the ANN inverse sensor model of the respective cascade in the structure of the sensor represented by (13). The block diagram of the ANN second-order section ][2 iC is shown in Figure 5 and the block diagram of the ANN first- order section ][1 jC is shown in Figure 6. 3.3. Sequential representation  The recovery of the input signal of the sensor (1) is implemented by its measured output signal processing on the basis of the ANN inverse sensor model. This model is represented as the sequential connection of the correcting filter and identical first-order sections [14]. Every such section is the ANN inverse model of a first-order LTI system with the following TF:   1 1 1 1   pT pW . (18) The value of the time constant 1T in (18) is set equal to such a value among the time constants jT1 in (1), that provides the proximity of the step responses of systems represented by (1) and (18). The TF )( pWcf of the correcting filter is the inverse TF of the sensor, which is supplemented with a certain number mnq  of TFs(18) to ensure the stability of the inverse model. The TF )( pWcf of the correcting filter is as follows: Figure 4. Functional block diagram of the ANN inverse model of the sensor  in the cascade representation.  Figure  3.  Block  diagram  of  the  ANN  direct  sensor  model  in  the  general representation.  ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 28                    mn m ki i k i iii n lj j l j jjj q s q scf pT pTpTpT pTpTpT K pT pWpWpWpW                     1 1 112 112 1 1 1 1 1 2 1 22 22 2 1 1 1 11 22 1 0 1 1 1 1   . (19) The functional block diagram of the ANN inverse sensor model in the sequential representation is shown in Figure , where the first-order section  11 TC is the ANN inverse model of the first-order LTI system described by (18). Generally speaking, the direct TF(18) can be represented in the discrete form (15) and the inverse one in the form (17). Therefore, the structure of the section  11 TC , that implements the inverse TF(18), can be represented, in its turn, as it is shown in Figure 7. However, taking into account that the order of the TF(18) numerator is always equal to zero unlike the order of the TF(15) numerator, the discrete analogue of the TF(18) can be represented as follows:   1 1 0 1 1   z zW   . (20) The inverse model of the system (20) is described by the following TF:   1 10 1 0 1 0 0 1 1 1 1 1 01 1 1 1 1                  zz z z zW         (21) where 0 0 1    , 0 1 1     . The block diagram of the ANN section  11 TC that implements the inverse model (21) is shown in Figure 8, where 0w and 1w are weights of the ANN section  11 TC . It should be noted that the structure above represents an ANN finite impulse response filter. The basic advantage of this structure is its guaranteed stability. Therefore, it ensures stability of the ANN inverse model of the sensor as a whole and provides stable the sensor input signal recovery. 3.4. Noise cancellation approach  In practice, the recovery of the dynamically distorted input signal of the sensor on the basis of its ANN inverse model is accompanied by the significant increase of the additive noise at the sensor output, as well as the internal noise of the ANN inverse sensor model. For the correct recovery of the input signal of the sensor it is expedient to extend the ANN inverse sensor model, taking into account the presence of the additive noise at the sensor output. This extension can be implemented as the additional low-pass filtration of the recovered signal by means of an increase of the order of the sequential sections  11 TC in the structure of the ANN inverse sensor model. The block diagram of the ANN d -order section  1TCd with the filtration of the recovered signal is shown in Figure 9. The discrete TF of the section  1TCd is as follows: Figure 7. Functional block diagram of the ANN inverse model of the sensor in the sequential representation.  Figure 8. Block diagram of the ANN section C1[T1].  Figure 5. Block diagram of the ANN second‐order section C2[i].  Figure 6. Block diagram of the ANN first‐order section C1[j].  ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 29    ddcd zwzwwzW   ...110 , (22) where dwwww ,...,,, 210 weights of the section. The cancellation of additive noise amplified during the sensor input signal recovery is implemented due to the presence of the internal filter in the structure of the ANN section  1TCd . The discrete TF of the internal filter in the structure of the ANN section  1TCd is described by the following equation:      zWzWzW cdfd 1 , (23) where  zW1 is the discrete analogue (20) of (18). It should be noted again that the structure of the section  1TCd and hence its internal filter  zWfd represents an ANN finite impulse response filter. Therefore, it ensures stability of the ANN inverse model of the sensor as a whole and simultaneously provides stable sensor input signal recovery and filtration. 3.5. Training procedure and training sets composition  The block diagram of the ANN section  1TCd training is shown in Figure 10. The procedure of the ANN section training (i.e. its weights dwwww ,...,,, 210 adjustment) consists in the minimization of the training error represented as the aggregate standard deviation for all 1N samples of the input training set  kh y between the target  khu and actual  khu* outputs of the ANN section  1TCd :        N k uu khkh N E 0 2*1 . (24) A criterion for the training sets composition is proposed. This criterion allows to evaluate the training sets length 1N . In order to implement the filtration ability the sinusoidal smoothing of the ANN section  1TCd step response was applied. The filter (23) bandwidth regulation is achieved by the section order d adjusting. On this basis, the samples of the target training set are composed according to the following equation (where T is the sampling period and Nk ...,,1,0 ):                          Nkd dk d kT khu ,1 0, 2 22 sin1 )(  . (25) Then, the input training set should be composed from the step response samples of (18) as follows:      11 /exp1/exp1 TTkTtkh ky  , (26) where Tktk  are discrete times for Nk ...,,1,0 . Suppose, provided 1 , starting with time TNTh  all the successive samples of the TF(18) step response lie within the following range:   1h . (27) Then, the training sets can be composed in accordance with (25) and (26). The sensor step response as the source signal for the input training set composition underlies the definition of the required length of training sets  kh y and  khu . This consequence follows from the analysis of (24) for the ANN section  1TCd training in terms of the problem considered. The limiting value of the training error, where the length of the training sets approaches infinity, is as follows: Figure 10. Block diagram of the ANN section Cd[T1] training.  Figure 9. Block diagram of the ANN section Cd[T1].  ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 30        2 00 2 0 * 2100 1 1 lim ...,,,,limlim                             d i i N k d i uiu N d NN wikhwkh N wwwwEEE (28) Therefore, under conditions defined by (27) the error of the ANN section  1TCd training will lie within the following range:          202 2 0 2 2 0 0 0 2 0 0 2 0 111 11 1 11 1 1                                                Ew w N w N w N d j i N k d i i N k d i i N k d i ihhE . (29) Thus, (29) shows the direct relationship between the deviation of the training error from its limiting value and the training sets length 1N . 4. RESULTS OF EXPERIMENTAL DATA PROCESSING  An algorithm for the recovery of dynamically distorted signals on the basis of the proposed ANN inverse sensor model was developed. In order to validate experimentally the efficiency of the model and the algorithm, a dynamic measurement of temperature was made. The step response of the thermoelectric transducer (thermocouple) «Metran-281» by heating it from 0 °С to 800 °С was obtained. The result of the experimental data processing at d = 66 in the form of the plots of the thermocouple measured output  ty and the thermocouple recovered input  tu* is shown in Figure 11. The obtained result shows that the dynamic measurement error after the correction decreased by 40 % in comparison with the initial state without any additional correction. In addition, the time of the dynamic temperature measurement decreased from sT = 306 s to dT = 60 s, that is more than 5 times. These evaluations validate the efficiency of the proposed ANN approach to dynamic measurement error reduction. 5. CONCLUSIONS  The ANN approach to the recovery of dynamically distorted signals effectively reduces the dynamic measurement errors caused by the inertia of the sensor and the additive noise at its output. The considered ANN inverse model of a sensor with filtration of sequentially recovered signal effectively improves the sensor dynamic behaviour due to deep mathematical processing of measurement data. A criterion for the training sets composition for the ANN inverse model adjustment is proposed. This criterion helps to evaluate the length of training sets on the basis of the required training error. The results of the experimental data processing validate the considerable improvement of the sensor dynamic behaviour due to the application of the proposed ANN approach to the dynamic measurement of the temperature. The future work direction is to develop the dynamic measurement error estimator on the basis of the ANN approach. It is expected that the additional information obtained from the error estimator will optimize the adjustment procedure of the ANN inverse model. REFERENCES  [1] V.A. Granovskii, “Models and methods of dynamic measurements: results presented by St. Petersburg metrologists, in: Advanced Mathematical and Computational Tools in Metrology and Testing X. F. Pavese, W. Bremser, A. Chunovkina, N. Fischer, A.B. Forbes (editors). World Scientific Publishing Company, Singapore, 2015, ISBN 978-9-81-467861-2, pp. 29–37. [2] A.N. Tikhonov, V.Y. Arsenin, Solution of ill-posed problems, V.H. Winston & Sons, Washington, 1977, ISBN 978-0-47- 099124-4. [3] Yu.P. Petrov, V.S. Sizikov, Well-posed, ill-posed, and intermediate problems with applications (inverse and ill-posed Problems), VSP International Science Publishers, Leiden, 2005, ISBN 978-9-06-764432-7. [4] S. Engelberg, A mathematical introduction to control theory, Imperial College Press, London, 2005, ISBN 978-1-86-094570-0. [5] E. Rosenwasser, R. Yusupov, Sensitivity of automatic control systems, CRC Press, Boca Raton, 2000, ISBN 978-0-84-932293- 8. [6] M. Eslami, Theory of sensitivity in dynamic systems: an introduction, Springer-Verlag, Berlin, 1994, ISBN 978-3-66- 201634-3. [7] A.L. Shestakov, Dynamic error correction method, IEEE Transactions on Instrumentation and Measurement, 45 (1996) pp. 250–255. [8] A.L. Shestakov, Theory approach of automatic control in dynamic measurements: monograph, SUSU Publishing, Chelyabinsk, 2013, ISBN 978-5-69-604468-2 (in Russian). [9] A.L. Shestakov, “Dynamic measurements based on automatic control theory approach”, in: Advanced Mathematical and Computational Tools in Metrology and Testing X. F. Pavese, W. Bremser, A. Chunovkina, N. Fischer, A.B. Forbes (editors). World Scientific Publishing Company, Singapore, 2015, ISBN 978-9-81-467861-2, pp. 66–77. [10] R. Bogacz, B. Krupanek, “Double channel dynamic error correction of methane sensor”, Proc. of 21st IMEKO World Congress, Aug. 30 – Sept. 4, 2015, Prague, Czech Republic, pp. 1764-1767. [11] Z. Zhang, Y. Li, “Correction of signal distortion caused by sensor’s dynamic characteristic”, Proc. of 21st IMEKO World Congress, Aug. 30 – Sept. 4, 2015, Prague, Czech Republic, pp. 1024-1029. [12] S. Haykin, Neural networks: a comprehensive foundation (2nd edition), Prentice Hall, New Jersey, 1999, ISBN 978-0-13- 273350-2. Figure 11. Result of experimental data processing.  ACTA IMEKO | www.imeko.org  November 2016 | Volume 5 | Number 3 | 31  [13] J.-S.R. Jang, C.-T. Sun, E. Mizutani, Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence, Prentice Hall, New Jersey, 1997, ISBN 978-0-13- 261066-7. [14] A.S. Volosnikov, A.L. Shestakov, “Dynamic measurements error correction on the basis of neural network inverse model of a sensor”, Proc. of 21st IMEKO World Congress, Aug. 30 – Sept. 4, 2015, Prague, Czech Republic, pp. 2082-2087.