Adv Syst Sci Appl 2018; 1; 110-131 Published online at http://ijassa.ipu.ru. Models of adaptation in dynamical contracts under stochastic uncertainty Mikhail. V. Belov1, Dmitry. A. Novikov2 1) IBS, Moscow, Russia E-mail: mbelov59@mail.ru 2) Trapeznikov Institute of Control Sciences, Russian Academy of Sciences, Moscow, Russia E-mail: novikov@tushino.com Abstract: In this paper, we establish new sufficient conditions for the optimality of lump-sum and compensatory incentive schemes (contracts) under stochastic uncertainty. In addition, we suggest and analyze dynamical models of principal’s and agents’ adaptation to changes in statistical characteristics of an external environment. Keywords: contract theory, incentive problem, stochastic uncertainty, adaptive behavior. 1. INTRODUCTION Incentive problems in organizational systems, i.e., stimulation of controlled subjects to choose certain actions for the benefit of a control subject, are studied in organizational control [12, 13] and contract theory [21, 28]. Within the latter, much attention is paid to the situations in which the results of activity of controlled subjects depend on their own actions and also on external random factors. If the participants of an organizational system have a repeated interaction with the course of time, then dynamical contracts arise naturally. These contracts are described using the methods of repeated games (in discrete time [10, 24, 27] or continuous time [22, 29]), see the surveys in [11, 23, 27]. However, in some situations the characteristics of an external uncertainty change with time, and hence it is necessary to develop and analyze models for a proper consideration of such effects by organizational system participants, including their efficient response. This paper is organized as follows. In section 2, we introduce a classification system for the models of contracts in organizational systems and consider the static models of contracts, particularly the model under additive uncertainty (subsection 2.4) and the simple agent model (subsection 2.5). For these models, we establish new sufficient conditions for the optimality of lump-sum and compensatory incentive schemes (contracts). Section 3 is dedicated to contracts in multiagent systems while sections 4 and 5 to the dynamical models of earned value and adaptation processes of organizational system participants to changeable characteristics of an external environment. 2. STATIC MODEL Consider an organizational system (OS) [[13]] that consists of a single control subject (Principal) and a single controlled subject (agent). The agent chooses a nonnegative action y ≥ 0. In combination with a realized value of an uncertain parameter (known as the state of nature   [0, ∆]), the action uniquely defines the result z = y –  of his activity. This setup is called the additive uncertainty model, as the uncertainty distorts the agent’s action MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 111 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) additively. Assume the agent’s cost c(y, r) depends on his action y and type r > 0 (a parameter that reflects the efficiency of the agent’s activity). Let c(y, r) be a strictly monotonically increasing smooth convex function in y that vanishes in the origin and also a strictly monotonically decreasing function in r. The agent’s cost representation as a function of two variables is standard in organizational control (e.g., see [5–13]). In most cases considered below, the agent’s type r is fixed and/or conceptually insignificant. For the sake of compactness, we will omit r in the cost function, using the notation c(y) whenever no confusion occurs. The Principal offers the agent to conclude a contract σ(z) that determines a nonnegative reward (and conditions to obtain it) depending on the achieved result. The agent’s goal function is the difference between the incentive and cost functions, i.e., ).,()(),),(( ryczzyf −=  (1) The Principal obtains an income H(z) from the agent’s activity (where H(∙) is a continuous nondecreasing function) and also bears incentive cost. In other words, the Principal’s goal function is the difference between the income and incentive functions: ).()()),(( zzHz  −= (2) Accept the following sequence of moves in this system, which is traditional for the incentive problems in organizational control [[2], [12], [13]] and contract theory [[21], [26], [28], [30]]. First, the Principal offers the agent a contract; second, the agent chooses his action in response to this offer and the payments are made accordingly. Note that the agent may reject the contract, choosing the zero action. In this case, he obtains no reward but bears no cost. Assume both participants of this organizational system seek to maximize their goal functions. Since the result of the agent’s activity depends on his action and also on the state of nature, the Principal and agent have to use all available information in order to eliminate the external uncertainty. There are the following types of uncertainty depending on the awareness of the system participants. – no uncertainty (the deterministic case in which ∆ ≡ 0, and this is common knowledge for the Principal and agent); – complete awareness (both subjects know the true value of the state of nature); – interval uncertainty (both subjects know merely a set of admissible values for the state of nature, i.e., a range [0, ∆]); – probabilistic uncertainty (both subjects know a probability distribution over the set of admissible values for the state of nature or for the result of the agent’s activity); – fuzzy uncertainty (both subjects know a membership function for an uncertain parameter defined on the set of admissible values). Denote by and <Φ(σ(∙), y)> the deterministic goal functions of the agent and Principal, respectively; these functions are obtained after elimination of the existing uncertainty for the state of nature. As a rule, an interval uncertainty is eliminated using the principle of maximum guaranteed result (see a survey of uncertainty elimination methods for incentive problems in [[12]]); a probabilistic uncertainty, using the principle of expected utility; a fuzzy uncertainty, the principle of maximally undominated alternatives. Designate as P(σ(∙)) the set of agent’s optimal actions that are implemented by the Principal within a contract σ(∙), i.e., )}.),(({maxarg))(( 0 yfP y =   Similarly, designate as M the set of admissible contracts. Accept the hypothesis of benevolence [[12], [13]], stating that the agent chooses the most profitable action for the Principal from the set of implementable actions. Then the incentive problem is to find an optimal contract σ*(∙), i.e., an admissible contract that maximizes the Principal’s goal function: 112 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) )}}.),(({max{maxArg)( )( * y PyM     (3) If the hypothesis of benevolence is not satisfied, the problem is to find an incentive scheme (contract) that yields the maximum guaranteed efficiency: )}}.),(({min{maxArg)( )( * y PyM g     Even for the model with one uncertain parameter, there are many possible combinations of the Principal’s and agent’s awareness (more specifically, the number of such combinations is 17 = 1 + 42, as the cases of nontrivial mutual awareness [[14]] are beyond the scope of this paper). Let us study some of them. 2.1. Deterministic case Here z ≡ y, and the solution of problem (3) is a lump-sum incentive scheme based on the principle of cost compensation for the agent [[1], [3], [13]]. It has the form 0 0 0 0 ( ) , , , ( ) 0, , C c x y x y x x  y       =    (4) where the optimal plan (the agent’s action desired by the Principal) is )}.()({maxarg 00 zczHx z −=  (5) Substituting (4) and (5) into (2) gives the Principal’s optimal payoff )}.()({max 00 zczHK z −=  (6) The corresponding value of the agent’s goal function is 0. If the hypothesis of benevolence fails, the solution is the ε-optimal incentive scheme 0 0 0 ( ) , , ( , ) 0, ;gC c x y x x y y x   +  =   (7) where ε > 0 represents an arbitrarily small constant. 2.2. Complete awareness of Principal and agent Assume that the agent and Principal know the realized value of the state of nature before choosing the action and incentive scheme. Then the Principal can use the so-called flexible planning mechanism [[9]] in which the optimal plan x*() = arg 0 max y [H(y – ) – c(y)] (8) and the incentive scheme * * * * ( ( )), ( ) , ( ( ), ) 0, ( ) , C c x z x x z z x          − =   − (9) both explicitly depend on the state of nature . The value of the agent’s goal function is 0 while the Principal’s optimal payoff takes the form K() = 0 max y [H(y – ) – c(y)]. (10) Obviously, as the agent’s cost function is nondecreasing, we obtain K() ≤ K0 0  . This means that the uncertainty has a negative effect on the Principal’s payoff. 2.3. Interval uncertainty If there is incomplete awareness in the system, we should discriminate between the cases of symmetrical (identical) and asymmetrical awareness of the Principal and agent. (A standard hypothesis is that the agent has at least the same awareness of the uncertain parameters as the Principal [[13]]. Hence, considering the case of asymmetrical awareness below, we assume that the agent knows the realized value of the state of nature while the Principal makes his decision under uncertainty). Note that the agent may report information to the Principal [[2], [13]], this setup is not studied here. MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 113 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Asymmetrical awareness. In this case, the Principal knows merely the admissible range [0, ∆] for the state of nature and hence is forced to guarantee cost compensation for the agent. xMGR = arg 0 max y [0 ] min ;  [H(y – ) – c(y)] = (11) = arg 0 max y [H(y – ∆) – c(y)]. The Principal uses the incentive scheme σC(xMGR, z) = MGR MGR MGR ( ), , 0 . c x z x , z x  −    −  (12) The agent knows the realized value for the state of nature  and chooses the action y*() = xMGR – ∆ + , (13) which leads to the expected result (xMGR – ∆) of his activity. The agent benefits from plan fulfilment, which is easy to check directly by comparing his payoffs. For any state of nature, the Principal’s optimal payoff K∆ = 0 max y [H(y – ∆) – c(y)] (14) under asymmetrical awareness is not higher than under complete awareness (see (10) and (14)). The agent’s goal function takes the value f(σC(xMGR, xMGR), y*(), xMGR) = c(xMGR) – c(xMGR – ∆ + ) ≥ 0 . (15) Value (15) is called informational rent, i.e., the payoff obtained by a subject (in our model, agent) owing to better awareness in comparison with other subjects (the Principal). Symmetrical awareness. In this case, neither the agent nor the Principal know the realized value of the state of nature at the moment of decision-making. The only available information is the admissible range [0, ∆]. Thus, the Principal uses the incentive scheme (12) and obtains payoff(14), whereas the agent is forced to choose the action guaranteeing a nonzero reward to him: yMGR = xMGR . (16) As a result, the agent has zero payoff. Note that the organizational system suffers from the overproduction (∆ – ) ≥ 0. 2.4. Probabilistic uncertainty: additive model Let the uncertain state of nature  be a random variable with a continuous distribution function F̂ (∙): [0; ∆] → [0; 1], which has a density function p(∙). In the sequel, we will use a distribution function Fθ(∙): (–∞; +∞) → [0, 1] of the form Fθ(ξ) = 0, 0, ( ), [0, ], 1, . F̂            Assume there is asymmetrical awareness of this uncertain parameter in the system. For a given agent’s action y, the result of activity z = y –  is a random variable with the distribution function Fz(∙, y): [y – ∆, y] → [0, 1] defined by Fz(q, y) = 1 – F(y – q) . (17) Let the Principal’s choice be limited to the parametric class of lump-sum incentive schemes σC(π, z) = , , 0, , z z       (18) where π ≥ 0 and λ ≥ 0 denote some parameters, π is a planned result (the result of the agent’s activity desired by the Principal). (As demonstrated in [[6], [12]], in the case under study the lump-sum incentive schemes can be not optimal, see the discussion below; nevertheless, they are simple and widespread in applications). Under an action y ≥ π chosen by the agent, the expected value of his reward (18) is 114 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) Ez σC(π, z) = λF(y – π) (19). Suppose the agent seeks to maximize his expected utility [[12], [13], [21]]. By the first- order optimality conditions, the agent’s action y*(π, λ) ≥ π satisfies the equation λ p(y *(π, λ) – π) = c(y*(π, λ)). (20) The Principal’s problem is to choose the parameters π ≥ 0, λ ≥ 0 of the incentive scheme (contract) in order to maximize his expected utility: .max)),(()()),(( 0,0 * 0 *   →−−−    yFdpyH (21) Example 1. Assume the Principal has a linear income function H(z) = γz, where γ > 0 is a given constant; the agent has a quadratic cost function c(y, r) = y2/2r, where r > 0 is his type [[13]], which describes the efficiency of the agent’s activity; the distribution F(∙) is uniform, i.e., F(v) = v/∆, v  [0, ∆]. According to formula (19), the expected reward of the agent is λ(y – π)/∆. Hence, for any action y ≥ π the agent receives the expected payoff Ez f(σC(π, z), y, z) = λ (y – π)/∆ – y2/2r. (22) Maximizing his expected payoff (22), the agent chooses the optimal action (also, see expression(20)) * for , 2 ( , ) 0 for . 2 r r y r            =      (23) The expected value of the Principal’s goal function is Ez Φ(σC(π, z), z) = γ(y – ∆/2) – λ(y – π)/∆. (24) Substituting (23) into (24), we obtain the following optimization problem for the parameters of the incentive scheme (18) (see (21)): γ (λr/∆ – ∆/2) – λ(λr/∆ – π)/∆ → 0, /(2 ) max r     . (25) The solution of problem (25) has the form λ* = γ∆, π* = γr/2. Then the expected payoff of the Principal is γ(γr – ∆)/2 while the expected payoff of the agent is 0. •* Let us design the optimal incentive scheme for the probabilistic additive uncertainty model. The general solution procedure of probabilistic incentive problems is as follows [[12]]. First, for each agent’s action x, find a minimal incentive scheme min(x, ) in terms of the expected incentive cost of the Principal that implements this action, i.e., the scheme stimulating the agent to choose x  P(min(x, )). Second, find the action that is most beneficial to implement in terms of the Principal’s goal function, i.e., the one maximizing his expected payoff (also see expression (3)): x* = arg 0 max x EΦ(min(x, ), x), (26) where E means the expectation operator. Denote x** = arg max x [ 0 ( ) ( )H x p d    − – c(x)]. Lemma 1: In the probabilistic incentive problem, for any agent’s action x ≥ 0 there does not exist an incentive scheme implementing this action with the Principal’s expected incentive cost strictly smaller than the agent’s cost, i.e., min(x, ) ≥ c(x). * Throughout the paper, the symbol “•” indicates the end of an example or proof. MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 115 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Proof. Assume on the contrary that there exists an agent’s action 0x  and an incentive scheme ( )z such that Ez ( )|z x = 0 ( ) ( )zz p v, x dv +  < ( )c x , (27) and the incentive scheme ( )z implements the action 0x  , i.e., for any y  0, Ez ( )|z x – ( )c x ≥ Ez ( )|z y – c(y). (28) For y = 0, by c(0) = 0 inequality (28) takes the form Ez ( )|z x ≥ ( )c x + Ez ( )|0z . This result contradicts (27), as the incentive and its expected value are nonnegative. • We will establish sufficient conditions for the optimality of an incentive scheme of form (18), namely, the contract σC(x, z) = ( ), , 0, c x z x z x .  −    −  (29) Proposition 1: For all x  [x** – , x**], let ** ** ( ) ( ) ( ) c x p x x c x   −  . (30) Then in the probabilistic additive uncertainty model the incentive scheme (29) implements the agent's action x** with the minimum expected incentive cost c(x**) of the Principal and is optimal. Proof. Calculate the expected value of the agent’s reward from choosing an action y given a plan x: Ez σC(x, z)|y = 0 C ( ) ( ), px  y d    −  − . It follows from (19) that Ez σC(x, z)|y = c(x) F(y – x + ∆). The agent’s expected utility is Ez σC(x, z)|y – c(y) = ( ), , ( )  ( ) ( ), [ , ], ( ) ( ), c y y x c y y x x c x c c x F y  x y y x     .  − −  −   −  −  −  +   Given the plan x = x**, by condition (30) the maximum value of this expected utility is achieved at the zero action or at the action coinciding with the plan x (condition (30) guarantees that the agent’s expected utility is a nondecreasing function in his action on the interval [x** – , x**]). On the strength of the hypothesis of benevolence, the agent chooses the action x**, which makes the expected incentive cost of the Principal equal to the agent’s cost. Hence, by Lemma 1 the incentive scheme (29) is optimal. • Obviously, if the hypothesis of benevolence fails, then under condition (30) the ε-optimal incentive scheme is σCg(x, z) = ( ) , , 0, c x z x z x . +  −    −  Example 2. For the data of Example 1, condition (30) takes the form γr ≥ 2∆. • To proceed, study sufficient conditions for the optimality of a compensatory incentive scheme. To this end, find an incentive scheme ( )ˆ z that nullifies the agent’s expected utility for any actions, i.e., ( ) ( ) ( ) y y ˆ z p y z dz c y − − = . (31) Proposition 2: If there exists a contract ( ) 0ˆ z  satisfying relationship (31) then it is optimal in the probabilistic additive uncertainty model. 116 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) The validity of Proposition 2 follows from property (31) of the incentive scheme ( )ˆ z and Lemma 1. Therefore, an incentive scheme ( )ˆ z is optimal if there exists a positive solution of the integral equation (31). Let us formulate the existence conditions of such a solution. Proposition 3: If the functions ( )yc y and ( )p v  are continuous on the domains of definition, then the integral equation (31) has a unique solution, which can be calculated using the sequential approximations        = − −  =  = + ....,2,1, )0( )( )(ˆ )0( )( )(ˆ , )0( )( )(ˆ 0 1 0 idu p uzp u p zc z p zc z z ii       (32) For this solution to be positive, a necessary condition is ....,2,1,)()(ˆ)( 0 =−  iduuypuyc y i  (33) Proof. Differentiation of the integral equation (31) yields ).()()(ˆ)(ˆ)()(ˆ)0( zcduuzpuzpzp z z =−+−−  −   (34) First, solve (34) for z  : ).()()(ˆ)(ˆ)0( 0 zcduuzpuzp z =−+    Write this expression as . )0( )( )(ˆ )0( )( )(ˆ 0 du p uzp u p zc z z     − −  =  (35) Equation (35) is a Volterra integral equation of the second kind. As is well-known (e.g., see [[17]]), under the accepted hypotheses it has a unique solution, which can be calculated using the sequential approximations (32). Condition (33) is necessary for the positive solution. Thus, we have completely solved the problem for z  . Based on (34), it is possible to obtain a similar solution for z  [j; (j + 1)], j = 1, 2, …: 0 ˆ{ ( ) ( ) ( )} ( ) ˆ ( ) ( ) . (0) (0) z c z p z p z u z u du p p         +  − − = −  Note that it suffices to prove the positivity of ( )ˆ z on the first interval only, as c'(z) + p()(z – )  c'(z) for the subsequent intervals. Example 3. Consider the uniform density function )(p defined by 0 1 0 1 1 for [ , ], ( ) 0 for [ ; ], a a p a a       =    where 0 < a0 and a1 = a0 + . Then (31) can be written as  − − = 0 1 ).,()( az az rzcdzu Differentiation of both sides leads to the functional equation (z – a0) – (z – a1) = с'(z). Since c(y) = 0 and c (y) = 0 for y  0, the solution of this equation is given by the functional series   = −=− 00 ).()( i izcaz MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 117 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Recall that )(c is continuously differentiable and convex. Hence, the sum of the functional series is positive and increasing, which agrees with the main requirement to the incentive scheme (). Particularly, for the data of Example 1, c(y, r) = y2/2r and c(y, r) = y/r for y > 0; then   = −−=− 00 ),)(()( i iziz r az  where )( is the Heaviside step function 1 for 0, ( ) 0 for 0. u u u   =   • 2.5. Probabilistic uncertainty: The simple agent model An alternative to the additive uncertainty model considered in subsection 1.4 is the so- called simple agent model [[3], [5], [6], [12]], in which the distribution function of activity results has the form ( ), , ( , ) 1, . z G q q y F q y q y  =   (36) Here G(∙): [0; +∞) → [0; 1] is a given distribution function such that G(0) = 0, with a density g(∙). Like in the additive model, the agent’s action defines the maximum possible result while the distribution G(∙) does not explicitly depend on the action. Denote by pz(q, y) the density function associated with distribution (36). If the Principal uses the lump-sum incentive scheme (18) and the agent chooses an action y ≥ π, the expected reward of the agent is λ(G(y) – G(π)). For the simple agent model, an analog of the first-order optimality condition (20) has the form λ g(y*(π, λ)) = с(y*(π, λ)). As proved in the book [[12]], compensatory incentive schemes are optimal in the simple agent model with a risk-neural agent. For this class of models, the following optimal incentive schemes were constructed in [[6]]: – for a risk-averse agent, the compensatory incentive schemes σK(z) = 0 ( ) 1 ( ) z c v dv G v  − ; (37) – for a risk-seeking agent, the lump-sum incentive schemes σC(x, z) = ( ) , , 1 ( ) 0, . c x z x G x z x   −   (38) Clearly, the incentive scheme (37) is nonnegative, increasing and convex. By analogy with [[6], Lemma 1], we may establish the following properties of the incentive schemes (37) and (38). Proposition 4: In the simple agent model, for any y  0 the incentive schemes satisfy the relationships 1) 0 ( ) ( , ) ( )K zq p q y dq c y + = , 2) 0 ( ) ( , ) ( )C zq p q y dq c x + = . At the conceptual level, relationship 1) of Proposition 4 means that for any action of the agent his expected reward (37) coincides with his cost to choose this action. Hence, under the incentive scheme (37) used by the Principal, any actions of the agent yield zero utility for him and make his choice indifferent. By the hypothesis of benevolence, the agent will choose the action that is most profitable for the Principal. 118 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) If the Principal adopts the incentive scheme (38), then the agent is indifferent between the zero action (reject of the contract) and plan fulfilment. For the agent’s expected utility to reach a unique maximum as the result of plan fulfilment, the Principal has to increase payments for plan fulfilment by an arbitrarily small positive value ε. Note that this incentive scheme is not optimal but ε-optimal. The next property can be verified directly. Proposition 5: In the simple agent model, for any x  0 the incentive scheme C  (x, z) = ( , ) , , 1 ( ) 0, , c x r z x G x z x +  −   (39) is ε-optimal, i.e., it implements the agent’s action x with the minimum expected incentive cost of the Principal. Regardless of the scheme used by the Principal (compensatory or lump-sum), his optimal plan for the agent is given by x* = arg 0 max y [ 0 ( ) ( ) y H z g z dz + (1 – G(y)) H(y) – c(y, r)]. (40) The first-order optimality condition for (40) has the form H(x*) + (1 – G(x*)) H'(x*) = c(x*, r). (41) Example 4. For the data of Example 1, let G(z) = z/(1 + z). Then it follows from (37) that 2 2 ( ) 1 2 3 K z z z r    = +    ; using formula (41) we calculate         − − + = 1 1 31 2 1* r r x   . • This paper considers the simple agent model with a risk-neural agent. Hence, choosing between the incentive schemes (37) and (38), we should give preference to the lump-sum incentive scheme, as (a) it is simpler and (b) its ε-optimal analog stimulates the agent to fulfill the plan (see Proposition 5). The main results of section 1 dedicated to static problems of contract theory are the analytical relationships (31) and(37), which allow to formulate and solve complex problems (particularly, the dynamical ones with changeable characteristics of agents and/or state of nature, e.g., the parameters of distribution). First, we will extend the model with one agent and additive uncertainty to the multiagent case (section 3). Then, we will generalize the static simple agent model to the case of several sequential action periods (section 4). 3. MULTIAGENT MODEL Consider an organizational system composed of the Principal and n subordinate agents with simultaneous and independent decision-making. Denote by N = {1, …, n} the agent set and by ci(yi, ri) = c(yi, ri) the cost function of agent i; as before, yi ≥ 0 specifies the action of agent i and ri > 0 is his type. Designate as Y = i i N y   the total action of all agents. Assume the Principal is interested in a total result X ≥ 0 of the activity of all agents with a probability not smaller than a given threshold α [0; 1]. The value α is called contract reliability [[4]]. For the additive uncertainty model, this condition takes the form Y ≥ X + n 1F − (α). (42) The value n 1F − (α) can be treated as payment for uncertainty in terms of agents’ activity. Consider the following problem. What are the optimal plans for actions? Using Proposition 1, we obtain that for each agent the expected incentive cost of the Principal coincide with the agent’s cost to choose a corresponding action (under the incentive scheme MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 119 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) (31) used by the Principal, each agent receives a constant expected payoff regardless of his action; hence, by the hypothesis of benevolence each agent prefers plan fulfilment). Since the cost functions of the agents are nondecreasing, in the optimal solution condition (42) holds as equality. Hence, optimal plan calculation is reduced to the constrained optimization problem      += →    −   Ni i Ni x ii nFXx rxc i ).( min),( 1 }0{  (43) Using the Lagrange method of multipliers, we easily establish the following result. Proposition 6: In the additive uncertainty model, the optimal plans {xi *} in the contract yielding a total result X ≥ 0 with a reliability α are given by * ix = c'–1(, ri), i  N, (44) where μ > 0 is the solution of the equation   −− += Ni ii nFXxrc ).(),( 11   (45) As the distribution function is strictly monotonic, a direct analysis of problem (43) leads to Proposition 7: In the additive probabilistic uncertainty model, the Principal’s minimum cost to implement a given total result of agents’ activity does not decrease for higher contract reliability. Example 5. Consider the Cobb–Douglas cost functions for the agents, i.e., c(y, r) = 1  yμr1–μ, μ > 1. It follows from (44) and (45) that * ix = i j j N r r   (X + n 1F − (α)), i  N . (46) Using the optimal plans (46), calculate the optimal value of the goal function: ( )    −  −         + = Ni Nj j ii r nFX rxc . )( ),( 1 1 *      (47) The right-hand side of (47) is not decreasing in α (see Proposition 5). The payment for uncertainty (the difference between (47) and the value of the goal function in the corresponding deterministic problem) constitutes 1 1 ( ( )) ( )j j N X nF X r       − −  + −  , not decreasing for higher contract reliability. • 4. EARNED VALUE MODEL As a matter of fact, the earned value model is widespread in project management, both in theory and applications. In this section, we show a connection between the suggested models of contracts and the earned value model. Consider the interaction of the Principal and one agent within the scope of a certain project (a sequence of discrete periods). By a period T0 (called the project completion time), the Principal has to implement a given total result X0 ≥ 0 of activity. Let the states of nature { t}t=1,2,… in different periods be independent random variables obeying the same distribution F(∙). Assume the Principal concludes the optimal contract ( )tˆ z with the agent 120 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) (see Proposition 2) that satisfies (31) and specifies the agent’s reward depending on the result of his activity in period t, where t = 1, 2, … . The agent’s type and cost function are independent of periods. Hence, under a given reliability α of each single-period contract, in each period the Principal has to allocate the same plan to the agent given by (48)x0 = X0/T0 + 1F − (α) (compare with formula (42)). By (31), the agent benefits from plan fulfilment. The total plan for the agent’s activity by period t makes up 0 tX = t x0 – t 1F − (α) = tX0/T0. (49) In terms of the earned value approach of modern project management [[7], [16]], sequence (49) is called the budgeted quantity of work scheduled (BQWS). Since the result z = x0 –   of the agent’s activity in period  is a random variable, the total result Xt achieved by a period t is also a random variable of the form Xt = tx0 – 1 t    =  = t(X0/T0 + 1F − (α)) – 1 t    =  = = 0 tX + t 1F − (α) – 1 t    =  .(50) Sequence (50) is called the actual quantity of work performed (AQWP). Now, introduce other indices of the earned value approach subject to the additive uncertainty model (t = 1, 2, …, T) [[7]] as follows. – the expected planned cost of the Principal, or the budgeted cost of work scheduled (BCWS), defined by 0 tc = tс(X0/T0 + 1F − (α), r); (51) – the actual cost of the Principal, or the actual cost of work performed (ACWP), defined by ct = 1 0 )( t ˆ x        = − ; (52) – work underrun (in terms of time, positive or negative), defined by δ(t) = min {δ | 0 t tX X− = }; (53) – earned value (EV), or the budgeted cost of work performed (BCWP) as the planned cost of actually performed work, defined by t ec = ( ) 0 t tc − ; (54) – the current forecast T(t) of project completion time defined by T(t) = T0 + ε(t); (55) – total planned cost, also called the budget at completion (BAC) or the budget cost (BC), defined by C0 = T0 с(X0/T0 + 1F − (α), r); (56) – the current linear estimate of total cost, defined by C(t) = T(t) ct / t; (57) – the actual project completion time, defined by T’ = min {t ≥ 0 | Xt ≥ X0}; (58) – the difference between the actual and budgeted cost, or cost overrun (CO), defined by ∆сe(t) = ct – ct e; (59) – schedule performance index (SPI), defined by at = t ec / 0 tc ; (60) – cost performance index (CPI), defined by bt = t ec /ct. (61) MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 121 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) The budgeted cost indices (48)–(61), which are traditionally divided into primary (48)– (52) and derived (53)–(61), are efficient tools for project management, at the stages of planning and implementation. Example 6. For the data of Example 1, let r = 1, T0 = 100, X0 = 100, ∆ = 1, and α = 0.2. The trajectories of the total plan (49), the total result (50) and the expected result Xt = 0 tX + t( 1F − (α) – E θ)) are illustrated Fig. 1. Simulation was performed in RDS software complex [[15]]. Fig. 1. Dynamics of results in Example 6 The dynamics of the planned and actual cost and also of earned value are shown in Fig. 2. The calculations yielded T’ = 145, BAC = 72, and ∆сe(T’) ≈ 60%. • Fig. 2. Dynamics of cost and earned value in Example 5 5. ADAPTATION MODEL Incentive problems in dynamical organization systems can be classified using different bases such as the relationship between periods, the foresight of system participants, the mode of decision-making and others, see [[11]]. In this section, we first introduce a classification of dynamical incentive problems with a single-time change of a model parameter, e.g., the Principal’s goal function, the agent’s cost function or the distribution function of activity results. Such a change occurs at a time td and will be further called a discord. By assumption, the system participants are short-sighted: in each period, they make decisions for this period only and do not consider the consequences in future periods. Then we study a model in which a discord occurs for the distribution function while the time of discord is unknown to the Principal and agent. The changes in the behavior of the system participants after Total plan (black), expected result (blue), and total result (red) Periods Planned cost (black), actual cost (red), and budgeted cost (blue) Periods 122 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) detection of the discord can be treated as their adaptation to the new operating conditions [[8], [18]]. 5.1. A classification of dynamical incentive problems In Table 1 we present all possible cases for the awareness of the Principal and agent about the new functions (income, cost, distribution) after a discord occurs. These functions will be denoted using appropriate symbols with tilde. Assume that, before choosing their decisions for a period t, the Principal knows the history Zt–1 = (z1, …, zt–1) while the agent the histories Zt–1 and Yt–1 = (y1, …, yt–1). The awareness of the Principal and agent are shown in columns 3 and 4 of Table 1. 1-2. A change of the Principal’s income function ( )( ~ )( zHzH → ) is considered in rows 1 and 2 of Table 1. If the Principal knows the new income function and the discord time td (row 1 of Table 1), then the problem is reduced to a set of typical static incentive problems studied in section 1, which are solved for each period independently. However, if the new income function )( ~ zH or the discord time td are not available for the Principal, the problem makes no sense: the Principal does not have enough information for decision-making. 3-9. A change of the agent’s cost function ( )(~)( ycyc → ) is considered in rows 3–9 of Table 1. If the agent and Principal both know the new cost function and the discord time (row 3 of Table 1), then the problem again is reduced to a set of typical static incentive problems, which are solved for each period independently. Assume the Principal is aware of the new cost function of the agent but has no information about the discord time (row 4 of Table 1). In this case, his rational behavior is to offer the agent a menu of optimal contracts for a set of possible cost functions. This is the screening principle used under asymmetric awareness [[13], [21]]). If the agent knows the new cost function and the discord time while the Principal neither of them (row 5 of Table 1), the problem makes no sense due to the following. For obtaining a positive payoff himself, the Principal has to stimulate the agent’s activity by offering a contract with a nonnegative payoff of the latter. Yet, being unaware of the agent’s cost function, the Principal cannot form such a contract. For the same reasons, the problems with the awareness described by rows 8, 12, and 15 of Table 1 are ill-posed: the Principal does not know the new cost function )(~ yc or the distribution function ),( ~ yzF . MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 123 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Table 1. Classification of dynamical incentive problems N o. Discord Agent knows Principal knows Problem 1 )( ~ )( zHzH → No matter td; )( ~ zH Typical 2 Nothing or td Ill-posed 3 )(~)( ycyc → td; )(~ yc td; )(~ yc Typical 4 )(~ yc Solvable by screening 5 Nothing Ill-posed 6 )(~ yc td; )(~ yc Typical 7 )(~ yc Typical 8 Nothing Ill-posed 9 Nothing No matter Ill-posed 1 0 ),( ~ ),( yzFyzF  → td; ),( ~ yzF td; ),( ~ yzF Typical 1 1 ),( ~ yzF Solvable by screening 1 2 Nothing Ill-posed 1 3 ),( ~ yzF td; ),( ~ yzF Typical 1 4 ),( ~ yzF D1 1 5 Nothing Ill-posed 1 6 Nothing No matter Ill-posed Let the Principal know the new cost function of the agent and the discord time while the agent merely this function (row 6 of 124 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) Table 1). This situation leads to a set of typical problem. Really, using complete information the Principal will offer a contract with c(y) before the discord and with )(~ yc after it; in turn, the agent can identify the discord time by the changing offers of the Principal and then respond optimally. Again, we obtain a set of typical static incentive problems. Now, consider the case in which the Principal and agent are both aware of the new cost function but the discord time is uncertain (row 7 of Table 1 MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 125 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Table 1). Recall that the agent’s cost function is continuously differentiable and strictly monotonic. Hence, observing his actual cost, the agent can detect the fact of discord (if any); more specifically, the agent reliably detects the change of the cost function at the end of the period after the discord, in which the agent chooses some action y such that )(~)( ycyc  . However, in this period the agent chooses his action without any knowledge of the cost function. Dealing with the short-sighted agent, the Principal has to offer a contract associated with the worst-case cost function for the agent, i.e., with the function ( ) max{ ( ); ( )}с̂ y c y c y= , before the discord and after it including detection by the agent. Thus, we arrive at a typical problem with additional cost of the Principal, which can be assessed by both players. If the agent does not know the new cost function, the problem is ill-posed regardless of the Principal’s awareness (row 9 of Table 1). In this case, he cannot estimate possible loss in several periods until the new cost function is identified. The agent prefers zero action accordingly. 10-16. A change of the distribution function of activity results or of the distribution function of the state of nature ( ),( ~ ),( yzFyzF  → ) is considered in rows 10–16 of Table 1. Assume the Principal knows the new distribution function ),( ~ yzF and the discord time while the agent at least ),( ~ yzF (rows 10 and 13 of Table 1). Then the problem again is reduced to a set of typical static incentive problems. Let the agent be aware of both the new distribution function ),( ~ yzF and the discord time and let the Principal be aware of ),( ~ yzF only. Here a sequential screening problem arises naturally (row 11 of Table 1). If the agent does not know the new distribution function ),( ~ yzF , the problem is ill-posed regardless of the Principal’s awareness (row 16 of Table 1). In this case, he cannot estimate possible loss or payoff in several periods until the new distribution function ),( ~ yzF is identified. The agent prefers zero action accordingly. Model D1 (row 14 of Table ) is a multiperiod contract model with a change of the distribution function ),( yzF at some time. In this model, the Principal and agent both know the new distribution function ),( ~ yzF and also expect a single change of this function. But they are a priori unaware of the discord time. Before period 1 of their interaction, the agent and Principal have no information about the discord except the prior. Hence, they have to act under the hypothesis that in period 1 the result corresponds to ),( yzF or ),( ~ yzF . Dealing with the short-sighted agent, the Principal has to offer a contract associated with the worst-case cost function for the agent (otherwise, the agent rejects the contract and both players do not receive new information about the state of nature, which makes further interaction unreasonable). If the Principal forms such a contract and the agent is rational, then the former can predict the action y of the latter. The result z is observed by the Principal, and hence the Principal has the same posterior information as the agent. This fact can be generalized as the principle of transparent stimulating contract and formulated in the following way. Assume the Principal can form a stimulating contract while the agent is rational and does not have strategic behavior; then the Principal can reliably predict the agent’s actions and use the same complete information as the agent. Hence, after period 1 the Principal can design a contract for period 2 using the prior knowledge about the functions ),( yzF , ),( ~ yzF and also using the observations Z1, Y1. In subsequent periods t, the Principal will act in the same way, forming contracts t(z) based on the available functions ),( yzF , ),( ~ yzF and the observations Zt–1, Yt–1. 126 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) For the short-sighted Principal, an alternative is to use an optimal contract given the distribution F(z, y) until he detects the discord (this situation is studied in detail below for model D1). Note that, owing to identical awareness, the Principal and agent detect the discord “simultaneously.” Depending on the availability of additional prior information for players, there may exist several contract design statements. If both players have prior information about the distribution of the discord times, it is possible to obtain a sequential optimal Bayesian algorithm to design the contract t(z). Finally, if both participants possess no additional information about the possible discord times and their interaction can be terminated in any period, then the minimax approach is optimal. 5.2. Discord problem (Problem D1) Consider the multiperiod simple agent model with unconnected periods [[10]] and let the Principal use the optimal incentive scheme (38) with the optimal plan (40). Assume that initially the Principal and agent have the same information about the distribution function G(∙). At some time td > 0 a discord occurs, which changes the distribution G(∙) to ( )G  . The new distribution ( )G  is a priori known to the Principal and agent, but none of them have information about the discord time. As the result of this discord, in a single period the expected utility of the agent varies by the value ∆f(G(∙), ( )G  ) = c(x*) * * * ( ) ( ) 1 ( ) G x G x G x − − . (62) Choosing in each period the action x*, the agent (as well as the Principal) observes the sequence of results Zt (also the agent observes the sequence Yt = (x*, …, x*)). Both players have to decide whether a discord occurs or not. The sequential problem is therefore decomposed into single-period problems with additional information (owing to independent periods). For period t, the agent and Principal renegotiate the contract using information about the possible distributions )(G , )( ~ G and the additional observations Zt–1. Define the value Lt = ln( ( , ) ln( ( , ))t t t tg z y g z y− to formulate the optimal sequential maximal likelihood rule for discord detection as follows. In each period t > 0, calculate lt (l0 = 0) by 1 1 1 0, if 0, , if 0. t t t t t t t l L l l L l L − − − +  =  + +  (63) If for some period lt > , then the discord takes place. Here the value  describes the error characteristics of the first and second kind. As is well-known [[19]], in comparison with other statistics the maximal likelihood statistic yields the most efficient decision rule in terms of the following criterion. One of the errors is detected with a probability not smaller than a given threshold while the other error is optimized. The threshold δ is chosen using the error characteristics of the first and second kind. Example 7. For the data of Examples 1 and 4, let G(z) = z/(β1 + z) and ( )G z = z/(β2 + z). Then Ez(y) = βi ln (1 + y/βi). Using condition (41), we find x* = 4 1 1 2 ( -1) r r r        − − −     . Expression (38) takes the form MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 127 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) σC(x*, z) = * 2 * * * ( ) ( ) , , 2 0, . x x z x r z x    +     (64) According to the principle of transparent stimulating contract, the agent always chooses the action         − − −−== 1 )1( 4 1 2 1 1*     yr r rxy . Before the discord time, the result of the agent’s actions obeys the distribution 2 1 1* * 1 1* )( )(),( z xz x xzg + +− + =      for z  [0, x*]; after the discord time, the distribution 2 2 2* * 2 2* )( )(),(~ z xz x xzg + +− + =      for z  [0, x*], where () denotes the delta-function. Then *12 1 2 * *2 1 * 1 2 ln 2ln for [0, ), ln 2ln for . t t t t z z x z L x z x x            + +      +     =     + + =    +    Take T = 500, td = 200, r = 1, γ = 10, β1 = 100, and β2 = 60. The dynamics of the planned, expected and actual results (in cumulative sums) are shown in Fig. 3 (dashed lines correspond to the dynamics without discord detection). The dynamics of the cumulative cost of the agent and the Principal’s incentive cost are illustrated in Fig. 4 (dashed lines correspond to the dynamics without discord detection). Fig. 3. Dynamics of cumulative results in Example 7 Planned result(black), expected result (blue), and actual result (red) Periods 128 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) Fig. 4. Dynamics of cumulative cost (agent, Principal) in Example 7 In the cumulative sum method, we may use the discord indicators defined by 1 1 ( ) t *tS t Ezz x  = = − (65) or * * 2 1 ( , ) ( , ) t t CS x z tc x r   = = − . (66) Their dynamics in our example can be observed in Fig. 5. Let us apply the maximal likelihood rule (63). The corresponding dynamics are presented in Fig. 6. The means of the statistic Lt before and after the discord are –0.04 and +0.04; the root-mean-square deviations are –0.29 and 0.27, respectively. Note that, before the discord, the statistic Lt takes the value * 2 1 * 1 2 ln ln 0.29 x x        + + = −    +    with the probability 1 * 2 0.5 x   = + ; after the discord, with the probability 2 * 2 0.38. x   = + Fig. 5. Dynamics of discord indicators S1 and S2 in Example 7 In our example, for δ = 2 the discord is detected after 10 periods since its occurrence. • Agent’s actual cost (black) and Principal’s incentive cost (red) Periods Discord indicators: S1 (red) and S2/100 (blue) Periods Discord indicator MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 129 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Fig. 6. Dynamics of discord indicator in Example 7 6.CONCLUSIONS In this paper, we have considered contracts between the short-sighted Principal and agents that operate under external probabilistic uncertainty (Knight’s measurable uncertainty [[25]]) with changeable characteristics (a reflection of Knight’s true uncertainty). A proper response to true uncertainty is a basic function of control subjects for the adaptive behavior of subordinate structural elements of activity [[1], [20]]. Among the promising lines of future research, we mention other descriptive methods for the influence of external uncertainty on activity results, contract renegotiation conditions for the long- sighted Principal and agents, and discord problems in multiagent dynamical organizational systems. REFERENCES [1] Belov, M. & Novikov, D. (2017). Strukturnye modeli kompleksnoi deyatel’nosti [Structural Models of Complex Activity]. Moscow, Russia: Fizmatlit [in Russian]. [2] Novikov, D. (Ed.) (2013). Mechanism Design and Management: Mathematical Methods for Smart Organizations. New York, USA: Nova Science Publishers. [3] Burkov, V., Gorgidze, I. & Ivanovskii, A. (1974). Prostoi aktivnyi element. Realizatsiya plana i pereotsenka busushchego sostoyaniya. Sintez funktsii dokhoda [Simple active element. Plan implementation and future state assessment. Income function design], in Aktivnye sistemy, ser. Problemy i metody upravleniya v aktivnykh sistemakh, vol. 2. Moscow: Institute of Automation and Remote Control, 52–62. [4] Burkov, V. & Novikov, D. (1997). Kak upravlyat’ proektami [How to Manage Projects]. Moscow, Russia: Sinteg, [in Russian]. [5] Burkov, V. (1977). Osnovy matematicheskoi teorii aktivnykh sistem [Foundations of Mathematical Theory of Active Systems]. Moscow, Russia: Nauka, [in Russian]. [6] Goubko, M. (2000). Zadacha teorii kontraktov dlya modeli prostogo agents [A problem of contract theory for simple agent model], Upravlenie Bol'shimi Sistemami, 2, 22–27. Periods 130 M.V. BELOV, D.A. NOVIKOV Copyright ©2018 ASSA Adv. in Systems Science and Appl. (2018) [7] Kolosova, E., Novikov, D. & Tsvetkov, A. (2000). Metodika osvoennogo ob’ema v operativnom upravlenii [Earned Value Approach in Operational Management]. Moscow, Russia: Apostrof [in Russian]. [8] Novikov, D. (2008). Matematicheskie modeli formirovaniya I funktsionirovaniya komand [Mathematical Models of Team Formation and Operation]. Moscow, Russia: Fizmatlit [in Russian]. [9] Novikov, D. (1997). Mekhanizmy gibkogo planirovaniya v aktivnykh sistemakh s neopredelennost’yu [Flexible planning mechanisms in active systems under uncertainty], Avtomatika i Telemekhanika, 5, 118–125. [10] Novikov, D. (1997). Mekhanizmy stimulirovaniya v dinamicheskikh i mnogoelementnykh sotsial’no-ekonomicheskikh sistemakh [Incentive mechanisms in dynamical and multiagent socioeconomic systems], Avtomatika i Telemekhanika, 6, 3–26. [11] Novikov, D., Smirnov, I. & Shokhina, T. (2002). Mekhanizmy upravleniya dinamicheskimi aktivnymi sistemami [Control Mechanisms for Dynamical Active Systems]. Moscow, Russia: Institute of Control Sciences [in Russian]. [12] Novikov, D. (1998). Stimulirovanie v sotsial’no-ekonomicheskikh sistemakh [Stimulation in Socioeconomic Systems]. Moscow, Russia: Institute of Control Sciences [in Russian]. [13] Novikov, D. (2013). Theory of Control in Organizations. New York, USA: Nova Science Publishers, 2013. [14] Novikov D. & Chkhartishvili A. (2014). Reflexion and Control: Mathematical models. London, UK: CRC Press. [15] Roshchin, A. (2012). Raschet dinamicheskikh system (RDS). Rukovodstvo dlya programmistov. Prilozhenie: opisanie funktsii i strukter. Prilozhenie k rukovodstvu dlya programmistov [Calculation of Dynamical Systems (RDS). Manual for Programmers. Appendix: Description of Functions and Structures. Appendix to the Manual for Programmers]. Moscow, Russia: Institute of Control Sciences [in Russian]. [16] A Guide to the Project Management Body of Knowledge (PMBOK® Guide), 5th ed. (2013). Project Management Institute. [17] Tricomi, F. (1957). Integral Equations. New York, USA: Interscience Publishers. [18] Tsyganov, V. (1991). Additivnye mekhanizmy v otraslevom upravlenii [Additive Mechanisms in Sectoral Management]. Moscow, Russia: Nauka [in Russian]. [19] Shiryaev, A. (1976). Statisticheskii posledovatel’nyi analiz. Optimal’nye pravila ostanovki [Statistical Sequantial analiz. Optimal Stopping Rules]. Moscow, Russia: Fizmatlit [in Russian]. [20] Belov M. & Novikov D. (2017). Reflexive Models of Complex Activity. Proc. of WOSC World Congress. [21] Bolton, P. & Dewatripont, M. (2005). Contract Theory. Cambridge: MIT Press. [22] Cvitanic, J. & Zhang, J. (2012). Contract Theory in Continuous-Time Models. Heidelberg: Springer. MODELS OF ADAPTATION IN DYNAMICAL CONTRACTS 131 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) [23] Horne, M. (2016). Essays on Dynamic Contract Theory. Ann Arbor: The University of North Carolina. [24] Ljungqvist, L. & Sargent, T. (2004). Recursive Macroeconomic Theory (2nd ed.). Cambridge: MIT Press. [25] Knight, F. (1921). Risk, Uncertainty and Profit, in Hart, Schaffner, and Marx Prize Essays, no. 31. Boston and New York: Houghton Mifflin. [26] Menard, C. (2000). Institutions, Contracts and Organizations: Perspectives from New Institutional Economics. Northampton: Edward Elgar Pub. [27] Renner, P. & Schmedders, K. (2016). Dynamic Principal-Agent Models, Swiss Finance Institute Research Paper no. 16–26. Zurich: University of Zurich. [28] Salanie, B. (2005). The Economics of Contracts (2nd ed.). Massachusetts: MIT Press. [29] Sannikov, Y. (2008). A Continuous-Time Version of the Principal-Agent Problem, Review of Economic Studies, 75(3), 957–984. [30] Stole, L. (1997). Lectures on the Theory of Contracts and Organizations. Chicago: Univ. of Chicago. http://restud.oxfordjournals.org/