Advances in Systems Science and Applications (2017) Vol.17 No.1 40 Control Cost in Adaptive Systems with an Identifier under Disturbance Description Uncertainties Alexander Bunich V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences, 117997, Profsoyuznaya street, 65, Moscow, Russia; E-mail: bunfone@ipu.ru Abstract. A problem of the criterion optimization of a linear-quadratic system with a stationary disturbance and set transfer function on control is considered. To solve the problem, a regularized method of substitution of estimates of the spectral density of the disturbance into the control cost functional is proposed. Using the non-regularized method of substitution is shown to be able to lead to deflate the control cost. Key words: linear-quadratic system, control cost, spectral smoothing of disturbances, substitution method. 1. SUBSTITUTION METHOD AND CONTROL COST A natural way of synthesis automatic systems with incomplete initial description is a reduction to a completely defined analog with substitution in the control law of corresponding empirical estimates instead of unknown parameters, calculated by an identifier. The identifier, as a sensor of parametric plant disturbances, may also be used to solve auxiliary problems, for instance, to predict slow faults, what demonstrates a universality of systems with an identifier. However, a justification of the substitution method, whose name was taken from the statistical estimation theory [1], requires considerable efforts. In the first turn, the optimization abilities are limited by two-block structures with separation of processes of forming the controls (identifier and controller with tuned parameters). Generically, restricting the class of strategies increases the control cost in comparison to the optimal (dual [2]) strategy. Besides that, the Bayesian scheme of recalculating a posteriori parameters distributions over past observations requires setting a priori distributions of these parameters. Finally, it is necessary to modify this scheme for problems with infinite control horizon [3]. The forced substitution of the initial problem statement of the synthesis with an asymptotic version implies considerable losses (a detailing of the term of ''asymptotic'' is needed: under increasing a sample, normalized risks may converge almost surely, in probability, in mean). Formally, the synthesis problem (of dual control) for a plant with a parametric uncertainty relates to the stochastic optimal control of a completely defined plant with partially observed state ),( tt xX  , extended by partial inclusion of unknown plant parameters  , and involves calculating a priori distributions of the parameter )/( t U ydp  with respect to the observations t t yyy ,...,( 1 ), where the sub-index U underlines the dependence of these distributions on selecting a control strategy )},({ 1tt t uyU (in problems with a scalar criterion, the class of selectors is enough). If the strategy selection does not influence the rate of studying (passive feed-back), then the filtering algorithm admits 41 A. Bunich: Control Cost in Adaptive Systems with an Identifier under Disturbance Description Uncertainties decomposition with emphasizing the parameters estimation in a functionally autonomous block (the identifier). Computational problems of the dual control are supplemented with inadequate assumption on known a priori distribution of the parameters, characterizing the so called ''a priori hardness''. A natural way to overcome this hardness is the asymptotic statement of the synthesis problem. Losses of the substitution are considerable. Firstly, there are not distinguished asymptotically optimal systems with different transient quality, so the problem statement needs detailing. Secondly, since a problem with infinite control horizon is considered, then a new constraint on the admissible strategies appears, which enables the closed-loop system stability, having no finite analogs. In systems with an identifier, investigating the stability is implemented within the method of ''frozen coefficients''. Correct using the heuristic principle of ''frozen coefficients'', excluding effects of the parametric resonance, is possible under artificial decreasing the rate of controller retune with respect to the identification rate). Asymptotically optimal strategies may be constructed without an identification by use of the direct method, as well as in systems with an identifier, when for a closed-loop plant conditions of the consistent estimation are violated (for instance, the Gauss-Markov conditions for the least squares method (LSM) [4]). Let us be distracted from a more detailed discussion of the place and role of the identification in control systems. We will be restricted by more weak purpose of the criterion optimization [5] (the criterion optimization problem is always solvable, involving cases, when the lower bound of the control cost in the class of admissible controllers is not achieved, and the problem of the argument optimization is unsolvable). Let us adopt the following assumptions. The plant is linear time-invariant, all variables are scalar and measured without disturbances; the transfer function of the plant on control is known; the disturbance 0}{  ttvv is centered stationary Gaussian random process with the spectral density of an unknown order ,Ks for which a consistent estimate ,...}2,1,{*  nKss n is known by use of the input/output observations  0},{ ttt yu , where K is a cone of spectral densities of stationary random processes of the autoregression/moving average with the 1L -metrics. The discrete-time ,...1,0t plant is described by the equation ttt vubya  )()( , 1)0( a with known coefficients of co-prime operator polynomials in the backward one time step shift operator . The initial data are disturbance independent random values with finite second moments. The controller (output feed-back) tt yu )()(   , 1)0(  is admissible, if the characteristic polynomial of the closed-loop system bag   is stable (the Schur one). The control cost is the output variance in the steady-state mode. It turns out, that under accounting even the simplifications listed the criterion optimization is rather complex and the ''natural'' substitution method may decrease the control cost. Let N be the completion of .K The cone N includes spectral densities of any stationary disturbances (indeed, K contains polynomials, in particular, convolutions of any density from N with the Fejer kernels of any orders, forming the approximative unit in 1L ). Thus, K may be Advances in Systems Science and Applications (2017) Vol.17 No.1 42 considered as a class of parametric approximations of Gaussian stationary random processes with absolutely continuous spectrum. On N , a concave functional ,||inf)( 20    dbfWI y f     is defined, where 0 yW is the transfer function from the disturbance to the output of the system with some fixed admissible controller,  is the ring of rational functions without poles in the closed disk }.1|{| z Due to the completeness of the affine parameterization of control systems by the functional parameter f [6] KssI ),( is the control cost, that is the lower bound of the control cost over the class of all admissible controllers. The substitution method to determine the control cost )(sI recommends the estimate *).(sI The recommendation is justified, if the cost functional is uniformly continuous: in this case, to small errors of estimating the spectral density, small errors of the estimation by use of the substitution method correspond. However, the cost functional meets only the more weak condition of the semi-continuity from below [7]. Indeed, let us continue uniformly the continuous cost functional from K on .N From the definition of the functional cost and the Runge theorem it follows that 0)( sI for any densities taking zero values on some arcs of the circumference 1|| z , and the set of such densities is dens in ,N so .0)( sI However, in contrast to the last said, the cost is strictly positive for any density Ks and the assumption on the uniform continuity is a delusion. In a number of cases, the designer possesses information on the spectral make-up of the disturbance in the form of the inclusion s with an a priori given class K . This information may be set by the inclusions ns . The paper is organized as follows. In Section 2, a class (of degenerated problems) is emphasized. Since the degeneracy is equivalent to the disturbance singularity, the control cost is zero. In Section 3, an example of decreasing the control cost under using the substitution method (a consistent estimate of the spectral density of a regular disturbance) is presented. In Section 4, a formula for the control cost is presented, and a procedure of the regularization of calculating the cost with using substitution of the metrics in the space of spectral densities of disturbances is proposed. Required reference information was presented in Section 1. 2. DEGENERATE PROBLEMS Synthesis problems with zero cost will be referred as degenerate. Let us resemble that the factorization method assumes the process to be regular, that is control system performance under the conditions of stationary inflow of new (renovating the disturbance) information from outside. A stationary process is regular if and only if its spectral density exists and has no ''deep zeros'' in the sense of convergence of the logarithmic integral: .ln 1Ls This a rather weak constraint does not even guarantee the Fourier series convergence for sln on a set of a positive measure (an example of A.N. Kolmogorov), in computational practice it is conventionally substituted by considerably more strict one of existing the spectrum (absolute convergence of this series). Correspondingly, the complex problem of the density s factorization is reduced to the separation for sln . A technical way of reducing the density factorization to separating its logarithm is complicated by the fact that the separation operator or ''natural projecting'' P ,  kj n k k kj n nk k eceсP            1 || , 43 A. Bunich: Control Cost in Adaptive Systems with an Identifier under Disturbance Description Uncertainties is not bounded in the 1L -norm. The high prefilter sensitivity to variations of the spectral make-up of the disturbances indicates that 1L -norm is not an adequate closeness measure; and possible way of regularization of synthesis of Wiener controllers is substitution of the metrics. From the category point of view, the regularity property is not typical: spectral densities meeting the regularity criterion forms in the space of all spectral densities the set of the first Baire category. The regularity (singularity) property is preserved under transforming the process by a stable filter. The factorization solves the inverse problem: determining the transfer function of the prefilter via the spectral density of the disturbance. These well known results admit a generalization to multivariate processes of a constant rank (with substitution in the criterion ss det ). For more general classes of stationary processes, effectively verified regularity criteria are not known, while determining the prefilter via the spectral density is a complex problem. A general theory of singular processes perhaps does not exist, however enough simple way is available to transfer from singular processes to regular ones by adding (mixing) the white noise of a small power (what is interpreted by ''measurement noise''). To pursue the purpose, one determines a prefilter that factorizes the density s with small 0 , reducing the case of singular disturbance to regular one (non-formal question of selecting  is solved experimentally). Under a fixed spectral density s , the control cost does not increase monotonically under as ,0 while solving the argument optimization problem is extremely non-regular and for the limit system (as )0 does not exist (let us resemble that for a singular disturbance the criterion optimization problem has zero solution). Forming control has a dual function: directing (minimizing the action risk) and studying (minimizing future risks due to an uncertainty non-removed) [6]. The adaptive systems with an identifier (ASI) are emphasized with their ability to parallelize these functions, what is seen from the structure ASI scheme with a functionally autonomous block, the identifier, calculating in the real time plant parameters estimates and controlling another block, the tuned controller, constructed at the stage of the synthesis of the main control system loop. Justification of the identification (indirect) approach to the synthesis problem is reduced to the fact, that if the control cost does not depend on transients in the identifier, then substitution of unknown plant parameters with their consistent estimates will not enlarge the control cost. 3.DECREASING THE CONTROL COST Let us show, by use of an example of a minimum-phase plant, that the substitution method may decrease the control cost [7]. In this example, the control cost coincides with the error variance of the optimal one step ahead prediction. The example idea is to approximate a density with the ''deep zero'' by spectral densities of regular processes. For a positive density Ks and a small number 0 , we will set s ( )()  s under ),(   and )exp()( 2 s under ).,(   The functions 1)( Ls  are bounded from zero and so ,)( Rs  where R is the class of spectral densities of regular processes. In accordance to the construction, ss L1  as ,0 however Advances in Systems Science and Applications (2017) Vol.17 No.1 44     dsln as 0 , and by virtue of the Szegő formula [8] ,0)( 0 2     where )(2  is the error variance of the optimal one step ahead prediction for the process with the spectral density s , coinciding, due to the minimum-phase plant property, with the control cost. Thus, for some family of regular processes the following conditions hold: , 1 ss L  Rs  (1a) ),(0)(lim 0 sIsI     (1b) Hence, the control cost takes as small as needed values in any neighborhood of the point .s One may prove that the latter affirmation is valid also without the constraints of minimum-phase and under any delays in the measurement. Lemma on decreasing the control cost. For any spectral density Ks , there exists a family of densities from K , meeting conditions (1). The lemma name is explained by the inequality 0)(inflim   IKV < ),(sI (2) where V is any neighborhood of the point .Ks Due to this inequality, 1L -metrics is not an adequate measure of closeness of disturbances in the spectral make-up, what is expressed by the incorrectness of the criterion optimization problem, possible decreasing the control cost, defined by the substitution method, in comparison to its magnitude under exactly known spectral distribution density. The problem regularization is possible by using additional information, or by introducing a new metrics in the class of spectral densities of regular disturbances. 4. CALCULATING THE COST AND SOLUTION REGULARIZATION The control cost )(sI has a simple geometric interpretation: it is the square of the distance from the point 201 LhWb yi  to 2H subspace. Its calculation enables one to regularize the criterion optimization problem by the metrics substitution: in the new metrics, the cost functional is slowly changed under small variations of the spectral make-up of disturbances. To calculate the control cost, let us introduce the canonical (interior-exterior factorization of the polynomial eibbb  , factorization *)2( 1hh of the spectral density ,s the distance )( from the point 2L to 2H subspace. Let us fix some admissible (support) controller and designate 00, uy WW the transfer functions of the control system with the support controller. Lemma. The control cost is defined by the formula )()( 012 hWbsI yi  . (3) Indeed, the distance d from the point hWy 0 to the 2Hbi subspace is equal to 2 0 2 | || |inf fbhW iyHf   , and due to the unimodular property of ib , the equality holds )( 0122 hWbd yi  . (4) But hbe is an exterior function, so the linear manifold hbe is dense in 2H , bh is dense in в 2bH , and 2d ,| || |inf 2 2WhMW (5) 45 A. Bunich: Control Cost in Adaptive Systems with an Identifier under Disturbance Description Uncertainties where M 0 yW .ib The value 2 2| || |Wh is the control cost of the control system with the spectral density s of the disturbance, and due to the completeness of the class M , 2d is the control cost, what was to be proven. Remark. As a support regulator, one may select the H -optimal regulator. On the code R of spectral densities of regular processes, let us introduce a new metrics ,| || |),( 22121 hhss  where 2,1h are solutions of the problem of the factorization of the densities Rs 2,1 with norming 0)0( h . It is easily to verify, that the mapping introduced as is a metric: ),0[: RR . Let ssn  R , for solutions hhn , of the problem of the factorization of corresponding densities , 2 hh H n  but  )(|||| ssssss nnn ,)(|| sshh nn  (6) where from due to the Cauchy inequality, we will obtain: Chhss nn 2 2 2 1 | || || || |  (7) with a large enough constant C . The limit transfer as n proves the following implication: ssn  R . 1 ss L n However, the inverse affirmation is mistaken. Let, for instance, the density s is a singular process density, while ns is its polynomial approximation, then .Rs The new metrics regularizes the criterion optimization problem, and in the new metrics on the class of spectral densities of regular disturbances the functional )(2/1 sI is a Lipschitz one. Let us underline, that the metrics introduced does not permit a continuation to more wide classes of densities, in contrast to the 1L -metrics, under definition of which the regularity condition is not pre-assumed. For rather poor classes of  , the regularization is not required. Example. Let  admit embedding in the simplex interior (parametric uncertainty). The continuity follows from the concave property. References [1] Borovkov A.A., Mathematical statistics, Nauka Publ., Moscow, 1984. (in Russian) [2] Feldbaum A.A., Foundations of the theory of optimal automatic systems, Nauka Publ., Moscow, 1966. (in Russian) [3] Fomin V.N., Methods of control of linear discrete-time plants, LGU Publ., Leningrad, 1985. (in Russian) [4] Kogan M.M. and Neymark Yu.I., “The Identifiability of locally-optimal adaptive control laws under indirect observations”, Automation and Remote Control, vol. 51, no. 1, 1990, pp. 53-62. [5] Tsypkin Ya.Z., “The Optimality in problems and methods of the advanced control theory”, Vestnik AN SSSR, no. 9, 1982, pp. 116-121. [6] Fomin V.N., Fradkov A.L., and Yakubovich V.A., Adaptive control of dynamic plants, Nauka Publ., Moscow, 1981. (in Russian) [7] Bunich A.L., “Degenerate linear-quadratic problem of discrete plant control under uncertainty”, Automation and Remote Control, vol. 72, no. 11, 2011, pp. 2351-2363. [8] Hoffman K., Banach spaces of analytical functions, Prentice Hall, Englewood Cliffs, N.J., 1962.