Microsoft Word - 977 Article Text, Copyedited.doc Adv Syst Sci Appl 2020; 03; 166-177 Published online at https://ijassa.ipu.ru. A Differential Stackelberg Game Theoretic Model of the Promotion of Innovations in Universities Mukharbek Malsagov1*, Gennady Ougolnitsky2, Anatoly Usov3 1) Ingush State University, Nazran, Russia E-mail: mmm1956@bk.ru 2) Southern Federal University, Rostov-on-Don, Russia E-mail: gaugolnickiy@sfedu.ru 3) Southern Federal University, Rostov-on-Don, Russia E-mail: abusov@sfedu.ru Abstract: A two-level dynamical game theoretic model "federal state - universities" in open- loop strategies is built and investigated. A refinement of the electronic learning courses, and their differentiation using modern methods of information technologies made by competing a la Cournot universities (agents) are treated as innovative investments. The algorithms of building Nash (in the non-coalitional game of agents) and Stackelberg (in the hierarchical game of the Principal with the grand coalition of agents) equilibria are proposed and implemented. For the solution of the respective dynamic control problems the Pontryagin maximum principle and simulation modeling are used. A comprehensive analysis of the received results is given. Keywords: Nash equilibrium, Stackelberg equilibrium, Stackelberg differential games, innovations, method of qualitatively representative scenarios in simulation modeling 1. INTRODUCTION A review of applications of the game theoretic models to the analysis of innovations is presented in [5]. The authors divide three levels of these applications: (1) intra-organizational level within a firm, where players are innovators, project managers, and resource administrators; (2) inter-organizational level where the players are competitive firms; (3) meta-organizational level, where players are a social planner (innovation policy maker, government, a social or government institution, e.g., a research foundation) and an aggregate innovative entrepreneur. A traditional approach to the building and investigation of the differential game models is exposed in [1,4]. An original method of solution of the Stackelberg differential games based on building of a mutually benefit program of actions and punishment in the case of deviations, is described in [9]. In our papers [13,16] the approach of [9] is extended for the case of several followers with consideration of the requirements of sustainable development of the controlled dynamic system [12]. Ougolnitsky and Usov [14] have proposed a method of qualitatively representative scenarios in simulation modeling. This method allows for a good enough qualitative forecast of the dynamics of a controlled system by means of few scenarios of the computer simulation. An interesting dynamic game theoretic model of the oligopolistic investment to the product differentiation is studied in [2, 3]. The paper [2] considers open-loop strategies, and the paper [3] analyzes closed-loop ones. Cellini and Lambertini [2, 3] have found that in the case of open-loop strategies the Nash stable volume of investments to the product differentiation (innovations) increases with a number of firms, i.e. the level of market * Corresponding author: mmm1956@bk.ru A DIFFERENTIAL STACKELBERG GAME THEORETIC MODEL… 167 Copyright ©2020 ASSA. Adv. in Systems Science and Appl. (2020) competition (Arrow hypothesis). In contrary, in the case of closed-loop strategies the dependence is inverse (Schumpeter hypothesis). The authors notice that innovation activity in the product differentiation can be considered as a contribution of private investors to the production of a public good. This fact embeds their paper to the context of public goods economics. Interesting models of Stackelberg oligopoly are proposed and investigated in [6- 8]. This paper develops the papers [2, 3] with consideration of other mentioned sources in the following directions. First, its problem domain is a development of the electronic learning courses in universities, where a modification of the courses is treated as an innovative activity. There is no doubt in the actuality of this problem. On March 25, 2020 the IT holding TalentTech and on-line universities "Netology" and EdMarket have presented the research results about the on-line education market†. As the authors notice, a volume of the Russian market of on-line education in the b2c segment was 38,5 billion rubles in the end of 2019. In the end of the year 2023, according to that forecast, its value will be equal to 60 billion rubles a year. The global market of on-line education up to the year 2023 tends to the amount $282,62 billion. An even higher estimate is given by the Interfax Academy. According to its report, the Russian market of the on-line education after 2019 equals to 45–50 billion rubles, and in 2020 will be equal to the amount 55–60 billion rubles, the annual growth is 20–25%. The global on-line education market is estimated by the value $74 billion (about 4,8 trillion rubles) after the year 2019, so the potential for Russia is great‡. It is evident that the growth of the distant forms of education due to COVID-19 pandemics will essentially enforce the noticed trends. Second, we propose a hierarchical setup of the problem with the federal state as a leader (Principal), and competitive universities as followers (agents). A description of a university as active system [11] is given in [10]. The hierarchical impact of the Principal to the agents may be administrative (compulsion) or economic one (impulsion). In the former case the Principal impacts to the sets of feasible strategies of the agents, and in the latter case - to their payoff functionals [12]. Thus, the contribution of the paper has four aspects. First, we consider a combination of the aggregative non-cooperative oligopolistic game of the agents with the Stackelberg game of the type "Principal-agents". Second, the parameter of the demand function varies in time, and the character of this variation depends on the agents' actions (strategies) in the form of a differential equation. Third, the agents choose both outputs and investments, i.e. an agent's strategy includes a parameter of its cost function called the constant cost. Fourth, the following approach is used. From the point of view of the agents, their interaction is modeled as a game in normal form where the Nash equilibrium is built. From the point of view of the Principal it is supposed that the agents form the grand coalition, and respectively a Stackelberg two-person game "Principal - coalition of agents" arises. This permits to avoid a challenging question about what should be considered as a best response of several agents to the Principal's strategy. The rest of the paper is organized as follows. In the Section 1 the setup of a dynamic problem of hierarchical control is given. In the Section 2 the Nash equilibrium for two symmetrical agents with administrative and economic impact of the Principal in open-loop strategies is built. Algorithms of building the Stackelberg equilibrium by means of the method of qualitatively representative scenarios in simulation modeling are presented. The Section 4 describes a numerical simulation in the problem of building the Stackelberg equilibrium. The Section 5 contains a comparative analysis of the received results, and the conclusions are formulated. †http://neorusedu.ru/news/rossijskij-rynok-onlajn-obrazovaniya-ozhidaet-burnyj-rost ‡https://www.kommersant.ru/doc/4275439 168 M. MALSAGOV, G. OUGOLNITSKY, A. USOV Copyright ©2020 ASSA. Adv. in Systems Science and Appl. (2020) 2. THE PROBLEM SETUP Let us consider the following hierarchical modification of the model proposed by Cellini and Lambertini [2, 3]. Several universities (agents) competing a la Cournot develop electronic learning courses for sale. A refinement of the courses, and its differentiation using modern methods of learning and information technologies are treated as innovative investments. The differentiation of the courses may be considered as a public good, and the respective investments - as a private production of the public good [3]. On the higher (relative to the agents) control level a Principal (federal state or its authorized bodies) is situated. The Principal tends to increase a public good (with possible additional consideration of its own interests) by means of administrative or economic control methods. In the case of administrative impact (compulsion) the Principal bounds from below the contributions of the agents to the innovative development (production of the public good) that incurs control cost. In the case of economic impact (impulsion) the Principal grants the agents on the base of an available budget. In both cases the Principal and the agents use open-loop strategies. The game is played on a finite interval of time In the case of compulsion the model for n agents has the form: - the Principal's payoff functional (2.1) - the Principal's control constraints ; (2.2) - the agents' payoff functionals (2.3) - the agents' control constraints ; ; (2.4) - the equation of system dynamics [9, 10] (2.5) - the current payoff function of the i-th agent (2.6) In the Principal's payoff function the profit πi reflects an approximate estimate of the positive externality from the universities' activity, namely, the GNP growth due to increasing of the educational level of the society; - the inverse demand function [2,3]; (2.7) - the payoffs of the Principal and the agents in the moment of time T respectively - the total payoff functionals of the Principal and the agents; - a symmetrical degree of substitutability between any pair of courses. If then the courses are completely homogeneous. If then the courses are unique and each agent is a monopolist [2,3]; - an output level of the i-th agent at constant returns to scale, then total operative cost per period are ; - individual investment of ].,0[ T max)()())()(( 0 0 1 1 0 ®+ú û ù ê ë é ÷ ø ö ç è æ--= ò å å = = - TGdttyZtsteJ T n i n i iii t pr max)(0 Ktyi ££ ;max)())()(( 0 ò ®++= - T iii t i TGdttsteJ pr max)()( Ktkty ii ££ max)(0 Qtqi ££ );( )(1 )( tD tK tK dt dD + -= ;)0();()( 1 BDtktK n i i ==å = )()(])([)( tktqctpt iiiii --=p å ¹ --= ij jii tqtDtBqAtp )()()()( ;,...,2,1),(])()()([)( niTqcTqTDTBqATG ii ij jii =---= å ¹ .)(])()()([)()( 11 0 å åå = ¹= ---== n i ii ij ji n i i TqcTqTDTBqATGTG iJJ ,0 ],0[)( BtD Î BtD =)( 0)( =tD )(tqi ),0(),()( iiiii ActqctC Î= )(tki A DIFFERENTIAL STACKELBERG GAME THEORETIC MODEL… 169 Copyright ©2020 ASSA. Adv. in Systems Science and Appl. (2020) the i-th agent to the innovative development, - the overall industry expenditure, - an upper bound for any ; - an upper bound for any ; functions are the Principal's grants to the i-th agent (in the case of compulsion they are given); T – the length of the game. Then, are the Principal's lower bounds established for - a discount factor; - demand parameters; - a convex increasing administrative control cost function, the function is supposed to be linear The equation of dynamics (5) is treated as a production function with the output created by the input . This technology can be shown to exhibit decreasing returns to scale w.r.t. . Thus is non-increasing function of time which tends to zero when [3]. In the case of impulsion the model has the form: - the Principal's payoff functional (2.8) - the Principal's control constraints ; (2.9) - the agents' payoff functionals (2.10) - the agents' control constraints ; . (2.11) Here are the Principal's control to be determined; is a total volume of the Principal's grants (with consideration of possible savings). All input functions of the model are supposed to be continuous, and the controls of the agents and the Principal belong to the class of piecewise continuous functions. We investigate the model (2.1)–(2.7) for compulsion and (2.5)-(2.11) for impulsion from the point of view of different control agents. From the point of view of the agents there is a non- cooperative n person game which solution is supposed to be a Nash equilibrium. From the point of view of the Principal there is a Stackelberg game. Let us assume in this case that the agents cooperate (create the grand coalition) and have the summary payoff functional in the form (2.12) Other model relations do not change. Thus, we have a Stackelberg game between the Principal and the grand coalition of agents. This game has the following information structure: 1. The Principal chooses its open-loop strategies for impulsion or for compulsion, . 2. Given these strategies for impulsion or for compulsion, the agents within the grand coalition choose their open-loop strategies , using (2.12). As an integrand function in (2.12) depends continuously on its arguments, and the domains of feasible controls of the agents (2.4) or (2.11) are non-empty closed sets, the problem of determination of the agents' best response to any Principal's strategy is resolvable. )(tK constmax =K )(tki constmax =Q )(tqi )(tsi )(tyi );(tki )1,0(Îr 0,0 >> BA )(×Z ;0)0( =Z )(×Z .cons;)( tEExxZ == )(/ tDD!- )(tK )(tK )(tD ¥®)(tK max)()]()([ 0 0 1 0 ®+-= ò å = - TGdttsteJ T n i ii t pr å = £³ n j ji Ststs 1 )(;0)( ò ®++= - T iii t i TGdttsteJ 0 max)())()((pr max)(0 Ktki ££ max)(0 Qtqi ££ )(tsi S .max)())()(( 1 01 ®÷÷ ø ö çç è æ ++== å òå = - = n i i T ii t n i iA TGdttsteJJ pr )(tss ii = )(tyy ii = niTt ,...,2,1],,0[ =Î )(tss ii = )(tyy ii = ),(),( tqqtkk iiii == niTt ,...,2,1],,0[ =Î 170 M. MALSAGOV, G. OUGOLNITSKY, A. USOV Copyright ©2020 ASSA. Adv. in Systems Science and Appl. (2020) 3. The Principal maximizes its payoff functional (2.1) or (2.8) for the worst best response of the coalition of agents to its strategy or . 4. The received set of strategies for impulsion or for compulsion is the Stackelberg equilibrium. 3. BUILDING THE NASH EQUILIBRIUM Let us first investigate the model from the point of view of the agents when they use open- loop strategies. The Principal's strategies are supposed to be given. This interpretation corresponds to the case of an indifferent Principal without its own objectives. Then we receive a differential n-person game (2.3)-(2.7) in the case of compulsion and (2.5) – (2.7), (2.10), (2.11) in the case of impulsion. Its solution is assumed to be a Nash equilibrium. To build it we use the Pontryagin maximum principle [15]. The Hamilton function of the -th agent both for compulsion and impulsion has the form: where is a conjugate variable (as function of time). From the necessary condition of extremum in the case of symmetrical agents ( ) we receive the system of equation for determination of their control variables (3.1) Thus . (3.2) Besides, we have the system of differential equations (3.3) From (3.1) we receive Therefore, the following proposition is proved. Proposition (3.1): The formulas (3.2), (3.3) determine the point of maximum of the Hamilton function for some value if the system (3.2), (3.3) has a solution and the values (3.2) belong to the domains of feasible controls (2.4) for compulsion or (2.11) for impulsion. If these conditions are not satisfied for some t then the Hamilton function attains its maximum for this at one of the bounds of the segments (2.4) or (2.11). 0J ),...,( 1 nssR ),...,( 1 nyyR ),...,,,...,,,...,( ** 1 ** 1 ** 1 nnn qqkkss ),...,,,...,,,...,( ** 1 ** 1 ** 1 nnn qqkkyy i ,))(1 ,)( )()()()()()),(),(),(),(( 1 1 1 å å å = - = ¹ + ++- ÷ ÷ ÷ ø ö ç ç ç è æ ---= n j j n i i iiiii n j ij jiiiii tk tkD ttsktqctqDtBqAttDttqtkH ll )(til ï ï î ïï í ì = = ¶ ¶ = ¶ ¶ ni q H k H i i i i ,...,2,1, 0 0 ;cci º niiGGqqkkssHH iiiiii ,...,2,1;;;,;;; ===ººººº ll ;0 )1( 1 2 =+ --= ¶ ¶ nk D k H l .0)()1()(2 =----= ¶ ¶ ctqnDtBqA q H ))1(2 )(;1)( -+ - = -+- = nDB сAtq n Dtk l ;11 2 2 ÷ ø ö ç è æ - -+÷ ø ö ç è æ + - -== ¶ ¶ - DDB CA dt d D H l ll ; )1)((2 ))(1( )( )( -+ -- = ¶ ¶ = nTDB Aсn TD GTl .)0(;11 BD Ddt dD = - +-= l .0;0)1(2; )1( 2 2 2 2 32 2 = ¶¶ ¶ <---= ¶ ¶ + = ¶ ¶ qk HnDB q H nk Dn k H l t t A DIFFERENTIAL STACKELBERG GAME THEORETIC MODEL… 171 Copyright ©2020 ASSA. Adv. in Systems Science and Appl. (2020) An analytical investigation of the system of equations (3.3) is impossible due to the form of the equation of dynamics (2.5) and presence of the third summand in (2.7) that is principal for the considered problem setup. Thus the system (3.3) was analyzed numerically by the method of shooting. The system of equations (3.3) has a solution not for all values of the model input parameters. The Picard theorem of existence and uniqueness of solution of the system of differential equations is not satisfied, the right hand sides are defined only for the negative values of λ(t)D(t). Besides, in the neighborhood of zero the Lipschitz conditions on λ and D are not satisfied. Therefore, the range of input model parameters for which the system (3.3) has the unique solution was determined numerically. A numerical identification of the model has a testing character and provides a reasonable relation of the model parameters and variables that allows for acceptable qualitative conclusions about the comparative analysis of the results of numerical modeling. Thus, the professors of Southern Federal University develop annually about 100 new learning courses, therefore the maximal possible value is taken An average cost of the development is equal to 50-80 thousand rubles, so (thousand rubles per year). The value of parameter B varied in the range (thousand rubles per year). We divided the cases of small investment of the Principal ( ), middle investment ( ), and considerable investment ( ), as well as different cases of the agents' control by the Principal: namely, the value of Z varied from 2 thousand rubles per year (soft control) till 10 thousand rubles per year (hard control). The discount factor was estimated as according to the annual inflation rate. The period of modeling was equal to 3 years, i.e. days for two universities ( ). Other parameters were evaluated by the experts (Table 3.1). Table 3.1. Test values of the model parameters Parameter Value 100 50 900 5 3000 0.04 600 2 Dimension - thousand rubles per year thousand rubles per year thousand rubles per year thousand rubles per year - thousand rubles per year - For determination of the range of model parameters in which the system (3.3) has the only solution we have implemented about 80 numerical simulations with variation of the values: (thousand rubles per year); n from 2 till 100. As a result it was established that for any input data there is a moment of time exists a solution of (3.3): . The moment belongs to the range from 950 till 1000 days and depends not essentially on the parameters . Its value slightly increases with n or B, decreases with A, and does not depend on c. Some of the results are presented in Table 3.2. Table 3.2. Dependence of the moment of time on model parameters 2 110 100 10 997 2 110 100 1 999 2 110 100 25 989 2 110 100 250 980 2 200 100 10 999 .100max =Q ,3000max =K ,900=А 50=c 151-=B 10/maxKnS = 5/maxKnS = maxKnS = 04.0=r 1095=T 2=n maxQ cci º A B maxK r ssi º Z ]200,10[Îc ]250,1[; ÎB ]1000,100[; ÎA :01 >t ),[0)(;0)( 1 TttanyfortDt γ- || )( 0 )( 0 ji JJ QRSuv l Ï)(),( QRSuv j Î)(),( D£- || )( 0 )( 0 jl JJ )( 000 ,, )()( lJJJ ji ),...,,,...,( )( 1 )( 1 )()( 10 )( 0 sss n ss uuvvJJ = 0;,, >D= ljis D ),( ),(),( max 000 0000 ),(),( ),( 00 vuJ vuJvuJ vuvu QRSvu - ¹ Î }, 2 ,0{)( maxmax n K n Kyty ii κ 0=iy n Kyi 2 max= n Kyi max= S },,0{)( )2()1( iiii sssts κ åå == == n i i n i i SsSs 1 )2( 1 )1( ; 2 0=is )1( ii ss = )2( ii ss = }, 2 ,0{ max max QQqi Î 174 M. MALSAGOV, G. OUGOLNITSKY, A. USOV Copyright ©2020 ASSA. Adv. in Systems Science and Appl. (2020) . For simplicity it is assumed that the strategies do not vary with time. Then we receive For small values the computer simulations are quite implementable. With consideration of the proposed strategies from QRS set the state variable is found analytically as 5. NUMERICAL CALCULATION OF THE STACKELBERG EQUILIBRIUM The computer simulations were implemented on personal computer with microprocessor A10 Intel Pentium G4620 with operative memory 4 Gb using an object oriented programming language C# according to the proposed algorithm. An average time of one simulation for the construction a QRS set was less than three seconds. The received results were analyzed by the following criteria: (a) summary discounted payoff of the Principal calculated by formulas (2.1) or (2.8); (b) index of system compatibility [12]: , where is the Principal's payoff in the Stackelberg equilibrium. This index demonstrates how necessary is Principal's presence in a control system. The closer is the value of to one, the more the system is compatible, and the less is a need in the hierarchical control by the Principal. For an initial QRS set the conditions (a) and (b) from the Definition 1 are checked. The value of is selected so that a difference between the Principal's payoffs for two any strategies from the initial QRS set does not exceed 10%. Then given the model parameters the second condition from the Definition 1 is checked. If necessary, the initial QRS set is extended or narrowed. In the case of extension the initial QRS set is added by new strategies. They correspond to the values situated between the previous ones (the strategies are supposed to be constant in time). Then the computer simulations are implemented. We have realized about 150 numerical calculations for three agents that form the grand coalition. The model parameters varied in the following range: for compulsion; for impulsion. In all cases the index of system compatibility is equal to one, and the system is completely compatible. Besides, in the case of impulsion on a short period of modeling (up to 4 years) both for the Principal and for the agents it is not profitable to invest to the development of innovative technologies (learning courses). Below the results of computer simulations for several sets of input data are presented. Let the grand coalition includes three agents, and days (4 years). Example 5.1: The values of model parameters are (thousand rubles per year). We varied the values The results for impulsion in the case are given in Table 5.1. In the majority of calculations . Table 5.1. Results of numerical simulations (Example 2) for impulsion }, 2 ,0{ maxmax n K n Kki Î .27nQRSm == 6