Adv Syst Sci Appl 2018; 03; 154-164 Published online at http://ijassa.ipu.ru/index.php/ijassa/article/view/568 Preliminary design with the epistemic uncertainty of parameters Georgy Veresnikov, Ludmila Pankova, Valeriya Pronina V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Scienses, Moscow, Russia E-mail: veresnikov@mail.ru Abstract: In design, especially in preliminary design, the assumption of parameter accuracy is not justified, since the parameters here are inaccurate (uncertain), due to insufficient knowledge or lack of statistics, as well as the fact that design parameters are further implemented in the production with some tolerance. The application of deterministic optimization methods under conditions of parametric uncertainty can lead to unacceptable solutions even with slight variation in the parameters. Currently, to account for uncertainty of the parameters there are commonly used stochastic methods designed to account for aleatory uncertainty with a priori known distribution functions of random parameters. However, in the preliminary design, most of the parameters are not random variables with known distribution functions. The necessary information on the parameters is obtained from the experts. In this paper, we develop methods and algorithms for preliminary design in conditions of epistemic uncertainty arising from lack of knowledge and observation results, replenished by expert assessments. In the paper the problem of optimal design in the presence of input and design parameters with epistemic uncertainty is considered. The choice of Liu's uncertainty theory for solving the problems of preliminary design is justified. The model of uncertain design parameter and optimization model with uncertain design and input parameters are proposed. The task of optimal design of the propulsion system parameters of supersonic maneuverable airplane is solved using the proposed models. The results are compared with the solution using Monte Carlo method. The solution time using the proposed model is two orders of magnitude less. Keywords: preliminary design, uncertain quantity, epistemic uncertainty, optimization under conditions of parametric uncertainty, deterministic duplicate, expected mean, model of uncertain design parameters, optimal design. 1. INTRODUCTION Preliminary design of complex technical systems is usually understood as an iterative multidisciplinary optimization process of searching for design parameters (solutions) with given objective functions and constraints that determine the admissible set of solutions. In the deterministic formulation of optimal design, the parameters are considered accurate. The influence of parameter uncertainty on the objective functions and on the admissibility of the solutions is not taken into account. In the design, especially in preliminary design of new technical objects, the assumption of parameter accuracy and the search for a global maximum in the deterministic formulation is not justified, since the parameters here are inaccurate (uncertain), due to insufficient knowledge, lack of statistics, as well as the fact that design parameters are further implemented in the production with some tolerance. In addition, multidisciplinary optimization involves independent optimization of subsystems, i.e. subsystems should be optimized without full information on the output (design) parameters of other subsystems, using expert estimates. Therefore, the search for design parameters in deterministic form can also cause inefficient solutions. The application of deterministic optimization methods under conditions of parametric uncertainty can lead to unacceptable solutions even with a slight variation in the parameters PRELIMINARY DESIGN WITH THE EPISTEMIC UNCERTAINTY OF PARAMETERS 155 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) [1]. The need to present and take into account uncertainty in engineering design is now universally recognized [2]. Any method of decision-making in design under uncertainty must provide three basic tools:  uncertainty representation model;  method of calculations with uncertain variables (method of propagation of uncertainty on a function of uncertain variables);  model of optimization taking into account uncertainty of parameters. In the paper a brief overview of these tools for decision-making in the conditions of uncertainty ("theories of uncertainty") is given. Unlike other uncertain theories Liu's uncertainty theory provides a tool for reducing optimization models with epistemic input parameters to models of mathematical programming (for a certain class of functions). This provides the possibility of using classical numerical or meta-heuristic optimization methods without the use of simulation and approximation schemes, which lead to unacceptable computational time, especially with significant number of parameters, and introduce an additional error. In Liu's theory input parameters are considered as uncertain. The work extends Liu's theory of uncertainty, namely, the model of uncertain epistemic design parameters and the optimization model with epistemic input and design parameters are proposed. The problem of preliminary design of supersonic maneuverable airplane parameters in the conditions of parametric epistemic uncertainty using the Liu's uncertainty theory [3] is solved. Section 2 provides a classification of uncertainty types and a brief overview of existing models of uncertainty. Section 3 gives an overview of the optimization models and methods under conditions of parametric epistemic uncertainty. The choice of Liu's uncertainty theory for solving the problems of preliminary design of a supersonic maneuverable aircraft is justified. In section 4 a model of epistemic design parameters is proposed and the expected mean formula for the objective function with epistemic design and input parameters is derived. In section 5 the problem of calculating the parameters of the propulsion system for supersonic maneuverable aircraft using the proposed model is solved. 2. CLASSIFICATION OF UNCERTAINTY AND MODEL OF EPISTEMIC UNCERTAINTY There are two types of uncertainty, depending on the available information: aleatory and epistemic. Aleatoric (objective) uncertainty is associated with a random variable for which there is statistical data to help make reliable conclusions about the distribution of a random variable. In this case, the uncertain quantity is modeled by the probability distribution function. The epistemic (subjective) uncertainty of the first kind is connected with the fact that there is no or not enough information on random variables in order to make reliable conclusions about the distribution of a random variable. In the literature, such uncertainty is called an imprecise probability. In this case, it is common practice to obtain information from experts. An imprecise probability is sometimes called second-order uncertainty: uncertainty about an uncertain variable, i.e. the uncertainty associated with either the subjectivity of expert information or the methods of processing incomplete data is added to the uncertainty of the random variable. To date, more than 20 "uncertainty theories" have been proposed for modeling imprecise probabilities [4], including evidence theory [5], possibility theory [6], interval analysis [7], interval probability (P-box) [8], the Liu's uncertainty theory [3]. Epistemic uncertainty of the second kind is associated with a nonrandom variable, the value of which is currently unknown. The expert gives information about the predictive 156 G. S. VERESNIKOV, L.A. PANKOVA, V.A. PRONINA Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) evaluation of this value. This kind of epistemic uncertainty is often called imprecise. The aleatory and epistemic uncertainties of the first kind can be classified as probabilistic form of uncertainty, while the second kind of epistemic uncertainty is non-probabilistic form of uncertainty. Possibility theory [6], fuzzy sets theory [9], interval analysis [7], uncertainty theory [3] are used for model non-probabilistic form of uncertainty. To represent the epistemic uncertainty, there are many models. However, there is no well-defined procedure for selecting an epistemic uncertainty representation model. A variety of models is discussed in [10]. In practice, the choice of the model is determined by the available information and the ability to obtain expert information about the uncertain parameters of the problem as well as the complexity of the objective functions, i.e. the complexity of computations with uncertain variable. 3. OPTIMIZATION MODELS AND METHODS UNDER CONDITIONS OF PARAMETRIC EPISTEMIC UNCERTAINTY. An objective function with uncertain parameters is an uncertain variable. In order to be able to compare uncertain objective functions and put the optimization task, it is necessary to introduce deterministic duplicates of objective functions, i.e. the numerical characteristics of these uncertain variables that will allow establishing the order relation on the set of uncertain objective functions. The choice of numerical characteristics as duplicates is the prerogative of the decision-maker (DM). The expected average value is most often used as the deterministic duplicate of the objective function. Perhaps, the decision maker will prefer to obtain a result corresponding to the smallest spread of the objective function. Then different spread indicators can be duplicates. DM also use a combination of both duplicates. DM may prefer to have a guaranteed result with a fairly high degree of belief. Then the guaranteed value of the function is used as a duplicate. In this case, the decision maker sets the level of belief that the values of the function will be less than the guaranteed value when minimizing the objective function and more than the guaranteed value when maximizing the objective function. In a rigid setting, constraints must be satisfied for any uncertain parameter values. In a more conservative formulation (soft constraints) the expert sets levels for the belief degree in the performance of constraints In a multicriteria formulation for several objective functions fi, i=1, …, m, a model with uncertain parameters can use different set of deterministic duplicates for each objective function [11]: min(max (D[f1],…, D[fm]), где D[fi] – set of deterministic duplicates for fi. Different aggregations of deterministic duplicates can be used as criteria. However, choosing a single metric for several objective functions is a very difficult task, which does not always have a solution. Therefore the aggregation of objective functions is rarely used in real multi-criteria optimization tasks. On the other hand, when the number of objective functions is greater than 2, there is a problem of visualization of multidimensional data. In this case, it is important to have tools for graphical representation of the Pareto front to facilitate the selection of solutions [12]. The problems of analysis and optimization with uncertain parameters usually have a large computational complexity, especially in applications to complex nonlinear systems. The application of epistemic parameter models using evidence theory, probability theory, fuzzy sets theory and interval representation often requires an excessively large number of samples and, as a rule, the found solution is not an extremum of the objective function. The calculation methods here rely on interval computations, which in general make optimization problems difficult to solve, in the worst case, it is not better than a brute force [13]. To PRELIMINARY DESIGN WITH THE EPISTEMIC UNCERTAINTY OF PARAMETERS 157 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) alleviate this problem, surrogate models were proposed [14]. However, the effectiveness of these approaches strongly depends on the accuracy of the surrogate model, and in addition the costs of building a surrogate model can be high. Calculation methods for these types of models, using Monte Carlo simulation methods, are easy to implement, but for engineering applications their computational cost becomes unacceptable. For engineering design in conditions of epistemic uncertainty, it is important to use mathematical formalism, which will be effective for computation and decision making. In this paper, an approach based on the Liu’s uncertainty theory was chosen to solve the problem of preliminary design. This theory for a certain class of functions ensures the reduction of optimization models with input epistemic parameters to models of mathematical programming. This allows to use classical numerical or meta-heuristic optimization methods without applying simulation and approximation schemes that lead to unacceptable computational time. In the future, uncertain variables will be called variables with epistemic uncertainty, modeled within the framework of Liu’s uncertainty theory. Let us describe a model of an uncertain variable in the uncertainty theory. The epistemic uncertainty of the event in uncertainty theory is the degree of the expert belief that the event will happen, i.e. measure of the uncertainty of this event M {*}. The measure of the uncertainty M is subadditive and the measure of the product of events is equal to the minimal measure of these events. Uncertain variable ξ is given (on the basis of expert estimates) by the distribution Ф(х) = M(ξ < x). M(ξ < x) is the degree of the expert belief that ξ is not greater than some value of x (measure of uncertainty). Unlike the measure of uncertainty, the probability measure is additive, and the measure for the product of events is equal to the product of the measures of these events. In particular, it is asserted in [3] that experts tend to overestimate the probability of cases leading to negative consequences, therefore the application of probability theory with epistemic uncertainty of parameters can lead to significant errors in the analysis. In [3] optimization models under conditions of parametric uncertainty are considered for uncertain input parameters. Let's give an analytical expression for the expected mean, which is used as a deterministic duplicate of an uncertain objective function. Let x be the vector of the design exact parameters,  be the vector of independent input uncertain parameters with inverse distribution functions Фi -1, i = 1, …, n, ),( xf be the objective function, ,0),( xg j j = 1, 2,…, p be constraints. If ),( xf is a continuous strictly increasing function with respect to ξ1, ξ2,…,ξm and strictly decreasing with respect to ξm+1, ξm+2,…, ξn, then:  dxfxfE nmm     1 0 11 1 11 1 ))1(),...,1(),(),...,(,()],([ 4. MODEL OF UNCERTAIN DESIGN PARAMETERS AND OPTIMIZATION MODEL WITH UNCERTAIN DESIGN AND INPUT PARAMETERS The need to take into account the uncertainty of the design parameters is caused by:  impossibility of providing accuracy of design parameter values in production conditions;  possible changes in the further iterative design process. Design parameters can take any value within a specified range. If we consider them to be uncertain, then for each such parameter an uncertainty distribution function must be specified. The definition domain of distribution function changes during optimization and does not exceed the range in which the solution is searched. To ensure the possibility of solving optimization problems, it is necessary to consider the uncertain design parameter in the form of uncertain variable, including deterministic and epistemic variable. The deterministic 158 G. S. VERESNIKOV, L.A. PANKOVA, V.A. PRONINA Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) variable provides the possibility of optimizing the objective function. The epistemic variable formed on the basis of expert information, takes into account the impressive of the design parameter. So, we consider that the uncertain design parameter x’ is modeled by the sum x’= x + δ, where x is deterministic design variable and δ is uncertain variable representing the impressive of this variable. The area for determining the uncertainty distribution function of δ depending on the preferences of DM can be given by absolute or relative limits. Let ],[ ba xx be an interval for x’, xa≥0, xb≥0. In the first case, the uncertainty distribution function of δ is defined on the interval [-δ a, δ b], δ a≥0, δ b≥0, δ b+δ a < xb- xa and x takes values from the range [xa+δ a, xb-δ b]. In the second case, the uncertainty distribution function of δ is defined on the interval [-kax, kbx], 0≤ka≤1, 0≤kb≤1, kbx+kax < xb-xa and x takes values from the range ] 1 , 1 [ b b a a k x k x  . In future we assume that the interval for δ is symmetric with respect to the mean value equal to 0, ka=kb, δ a=δ b. This option is most natural for expert. We derive an analytic expression for the expected mean, which is used as a deterministic duplicate of uncertain objective function. Let ' ix be independent uncertain design parameters, xi be deterministic design parameters, δi be independent uncertain input parameters with inverse distribution functions 1 i , i = 1, …, p, and ξj, j = 1, …, n, be independent uncertain input parameters with inverse distribution functions 1 j . If ),( 'xf is a continuous strictly increasing function with respect to ' 1x , …, ' qx and strictly decreasing with respect to ' 1qx  , …, ' px , also a continuous strictly increasing function with respect to ξ1, …,ξm and strictly decreasing with respect to ξm+1, …, ξn, then the expected mean for ),( 'xf will have the form:    d))(),...,(),(),...,( ),(x),...,(x),(x,...,)(x(f)],x(f[E nmm p1qq ' pqq          11 11 11 1 11 1 111 1 0 1 1 11 (1) 5. CALCULATING THE PARAMETERS OF THE PROPULSION SYSTEM FOR SUPERSONIC MANEUVERABLE AIRCRAFT We consider the task of preliminary design (PD) of a supersonic maneuverable aircraft, namely the calculation of aircraft propulsion system parameters ensuring the satisfaction of the requirements for the range of supersonic cruise flight (SCF) and priority tactical and technical requirements (TTR) in the subsonic region under conditions of parametric uncertainty is considered [15, 16]. Traditionally, PD is decomposed into several different disciplinary stages, where the parameters corresponding to the discipline are determined with the other parameters fixed, which ensure the satisfaction of TTR. Then, the iterative process of multidisciplinary optimization takes into account the links between several stages and different TTR. Since TTR are often contradictory, the choice of interrelated values of the aircraft design parameters, which ensure a compromise between the requirements of different disciplines is carried out. The values of the design parameters are formed as the results of solving various multicriteria optimization problems TTR to supersonic maneuverable aircraft contain requirements for subsonic and supersonic characteristics. At the preliminary design stage, the layout and propulsion parameters that satisfy the priority TTR on subsonic are first determined. Then the calculation of the parameters of the propulsion ensuring the performance of TTR on supersonic is performed with the fixed parameters PRELIMINARY DESIGN WITH THE EPISTEMIC UNCERTAINTY OF PARAMETERS 159 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) ensuring the performance of TTR on subsonic. The main requirement for supersonic modes is the requirement for the range of supersonic cruise flight (SCF), which depends to large extent on the parameters of the propulsion system in the SCF mode. Maximizing the range of the SCF for a given Mach number is equivalent to maximizing К/ce, where K is the aerodynamic quality of the aircraft, ce is the specific fuel consumption factor. The indicator of the level of wave resistance kw is a complex parameter reflecting the subsonic and supersonic aerodynamic characteristics of aircraft. An aircraft with a high value of kw is easier to construct and to ensure the performance of subsonic TTR. However, as kw increases, the maximum range of supersonic flight decreases. Thus, the problem of finding compromise solutions for choosing aircraft propulsion system parameters arises that provide the largest possible range and the largest possible value of the level of the wave resistance. Supersonic cruising flight is considered with Mach number M = 1.5 at flight altitude H = 11 km. This task of calculating parameters is presented as a two-criteria problem of finding Pareto solutions with criteria К/ce и kw [15, 16]. , ))(( )( dryopt P edryeopt weteqvfwoptosw e PPccP qSCkPqkl с K                      1 2 2 P e drye dryopt opt eqvfwet opt0R w c c PP P qCS РP k ,)( optdryreheatdryopt STCPPPP  STC = 0.4, where l is the wingspan, q is the velocity pressure magnitude at a given altitude and speed, kosw is the Oswald coefficient, dryP is the relative thrust of the engine in the "maximum" mode, reheatP is the relative thrust of the engine in the "full afterburner" mode, P0R is the prospective thrust of the engine in the mode of "full afterburner" (H = 0, M = 0), STCopt is supersonic throttling factor, corresponding optР , eqvfC is the coefficient of equivalent friction, Swet is surface area of the aircraft surface, dryec is specific hourly fuel consumption at the engine operating mode "maximum", P ec is slope of specific supersonic throttle characteristic. It is believed that the parameters of the geometry Swet, l and the prospective thrust P0R , which ensure the performance of the main (subsonic) TTR, were previously calculated. In conditions with uncertain design and input parameters, the two-criteria maximization problem with duplicates of the expected average will have the form: max (Е[К/ce], Е[kw]). The results of calculations for three optimization models are presented in the paper. In the first model the input parameters eqvfC , kosw are considered uncertain. In the second model the design parameters reheatP , dryec , P ec are considered uncertain. In the third model, the input parameters eqvfC , kosw and the design parameters reheatP , dryec , P ec are considered uncertain. For uncertain parameters linear uncertainty distribution functions are given. The linear distribution function of uncertainty, reflecting the absence of expert preferences in the range of values of an uncertain parameter, can be regarded as corresponding to the uniform distribution of a random parameter. This makes it possible to compare results obtained using 160 G. S. VERESNIKOV, L.A. PANKOVA, V.A. PRONINA Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) the theory of uncertainty and statistical multivariate modeling, without additional transformations. Geometry parameters and characteristics of the engine correspond to the aircraft type F/A-22. Multicriteria genetic algorithm was used to calculate optimal parameters. The following parameters of the multicriteria genetic algorithm from the Matlab Optimization Toolbox were used. The population size is 10,000 chromosomes, the number of generations is 400. The remaining parameters were left by default. Fig. 5.1 shows approximation of Pareto fronts for the model with uncertain input parameters for different ranges of uncertain input parameters. Using the Jensen’s Inequality [3], it is easy to show that depending on the convexity/concavity of the objective function relative to the input parameters, the Pareto front for the model with uncertain input parameters can move up/down relative to the Pareto front for deterministic model with nominal values of the input parameters. The blue Pareto front corresponds to the calculation with the exact nominal values of the parameters. Fig. 5.1. Approximation of the Pareto fronts for different ranges of uncertain input parameters Consider the second model. The first objective function eс K is strictly decreasing with respect to reheatP , dryec , P ec . In accordance with formula (1) ][ eс K E has the form: PRELIMINARY DESIGN WITH THE EPISTEMIC UNCERTAINTY OF PARAMETERS 161 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) , ))))(21(()21(( )( ][ 1 0    d PPckckccP qSCkPqkl с K E dryopt P e P edryedryeopt weteqvfwoptosw e     where optdryreheatreheatdryopt STCPPkPPP ))21((   . The second objective function wk is strictly decreasing with respect to dryec and strictly increasing with respect to reheatP , P ec . In accordance with formula (1) E[ wk ] has the form: ,1 )12( )21( 2 2 ][ 1 0    d ckc kcc PP P qCS РP kE P e P e dryedrye dryopt opt eqvfwet opt0R w                     where ,))12(( optdryreheatreheatdryopt STCPPkPPP   STC = 0.4. Optimization of deterministic parameters reheatP , dryec , P ec is performed in intervals ] 1 ; 1 [ k P k P b reheat a reheat  , ] 1 ; 1 [ k c k c b drye a drye  , ] 1 ; 1 [ k c k c bP e aP e  , where ];[ b reheat a reheat PP , ];[ b drye a drye cc , ];[ bP e aP e cc are intervals of change of uncertain design parameters. Fig. 5.2 shows the Pareto fronts approximation for the model with uncertain design parameters for different ranges of their variation. Fig. 5.2. Approximation of the Pareto fronts for different ranges of uncertain design parameters 162 G. S. VERESNIKOV, L.A. PANKOVA, V.A. PRONINA Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) From Fig. 5.2 it can be seen that Pareto-fronts in the calculation with different ranges of uncertain design parameters are shifted in the direction of the worst values of the objective functions relative to the Pareto-front for a deterministic model with nominal values of design parameters, i.e. solutions for the model with uncertain design parameters will always be more conservative. This task was solved using the Monte Carlo method to compare. The random variables reheatP , dryec , P ec were modeled in the intervals [-k reheatP ; k reheatP ], [-k dryec ; k dryec ], [-k P ec ; k P ec ] with a uniform distribution. Fig. 5.3 shows the Pareto-fronts approximation for calculating deterministic equivalents using the Monte Carlo method. Fig. 5.3. Approximation of Pareto fronts obtained by the Monte Carlo method Pareto fronts obtained from the second model with uncertain design parameters and obtained by the Monte Carlo method practically coincide. The Monte Carlo calculation time can be several orders of magnitude greater. This is because in the model with uncertain design parameters at each iteration of the genetic algorithm the expected mean of the functions K/ce, kw are calculated once. In Monte Carlo method a large number of computations of the functions K/ce, kw are required with the subsequent determination of the mean values E[K/ce], E[kw] (depending on the required accuracy). In solving this task by the Monte Carlo method there are 2000 calculations of K/ce, kw at each iteration. Fig. 5.4 shows approximation of Pareto fronts for the model with uncertain input and design parameters for different ranges of their changes. PRELIMINARY DESIGN WITH THE EPISTEMIC UNCERTAINTY OF PARAMETERS 163 Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) Fig. 5.4. Approximation of Pareto fronts for various ranges of uncertain design parameters with a range of uncertain input parameters 15%. By calculations of the model with uncertain design and input parameters it is impossible to estimate the influence of each type uncertainty parameters (input and design) on the optimization results, so it is advisable to carry out calculations on all three models. 6. CONCLUSION The paper substantiates the choice of Liu’ uncertainty theory for solving the problem of preliminary design for epistemic indeterminate parameters from the point of view of computational costs. A model of uncertain design parameters with epistemic uncertainty and an optimization model with uncertain design and input parameters are proposed. Using the proposed model, the task of calculating the propulsion system parameters for the supersonic maneuvering aircraft is solved. A comparison is made with the results of deterministic optimization. The results are presented for three optimization models under conditions of parametric uncertainty: with uncertain input parameters, with uncertain design parameters, with uncertain design and input parameters. The results are compared with the Monte Carlo method. Solution time using the proposed model is two orders of magnitude smaller. Further research is planned to focus on the development of optimization methods for non-monotonic functions, as well as robust optimization models. 164 G. S. VERESNIKOV, L.A. PANKOVA, V.A. PRONINA Copyright ©2018 ASSA. Adv. in Systems Science and Appl. (2018) REFERENCES [1] Nemirovski A. (2012) Lectures on Robust Convex Optimization. H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology. Atlanta Georgia. [Online]. Available http://www2.isye.gatech.edu/~nemirovs /RO_LN.pdf [2] Oberkampf W.L., DeLand S.M., Rutherford B.M., Diegert K.V., Alvin K.F. (2002) Error and uncertainty in modeling and simulation. Reliab. Eng. Syst. Saf., 75(3), 333– 357. [3] Liu B. (2007) Uncertainty Theory. 2-rd edition. Berlin: Springer-Verlag. [4] Zadeh L. (2008) Computation with imprecise probabilities. Proceedings of IPMU’08. Málaga, Spain. [5] Shafer G. A. (1976) Mathematical theory of evidence. Princeton, NJ: Princeton University Press. [6] Dubois D, Prade H. (1988) Possibility theory: an approach to computerized processing of uncertainty. New York: Plenum. [7] Jaulin L, Kieffer M, Didrit O, Walter E. (2001) Applied interval analysis. New York, NY: Springer. [8] Ferson, Scott, Vladik Kreinovich, Lev Ginzburg, David S. Myers, Kari Sentz (2003) Constructing Probability Boxes and Dempster-Shafer Structures. SAND2002-4015. Sandia National Laboratories. [9] Zimmerman H-J. (2001) Fuzzy Set Theory and Applications. 4th Rev. ed. Boston: Kluwer Academic Publishers. [10] Klir G.J. (2006) Uncertainty and Information: Foundations of Generalized Information Theory, Wiley-Interscience, Hoboken. [11] Veresnikov G.S., Pronina V.A., Pankova L.A. (2017) Uncertain programming in preliminary design of technical systems with uncertain parameters. Proceedings of the 12th International Symposium "Intelligent Systems". Atlanta: Elsevier, 103, 36-43. [12] Lotov, A.V., and K. Miettinen. Visualizing the Pareto Frontier. In J.Branke, K.Deb, K.Miettinen, R.Slowinski (eds.), Multiobjective Optimization. Interactive and Evolutionary Approaches, Lecture Notes in Computer Science, 5252, Springer, Berlin-Heidelberg, 213-244. [13] Sharyy S. P. (2018) Konechnomernyy interval'nyy analiz Institut vychislitel'nykh tekhnologiy [Finite-measured interval analysis] Institute of Computational Technologies, Novosibirsk: Publishing house XYZ, [in Russian]. [14] Helton J.C., Johnson J.D., Oberkampf W.L. (2004) An exploration of alternative approaches to the representation of uncertainty in model predictions. Reliability Engineering and System Safety, 85, 39–71. [15] Veresnikov G.S., Pronina V.A., Pankova L.A., Ogorodnikov O.V. (2017) Solution of preliminary design problems under conditions of parametric uncertainty. Control Sciences, 4, 65-73, [in Russian]. [16] Bashkirov I.G., Irodov R.D. (1997) Calculation of jet aircraft parameters under design requirements. World Aviation Congress, Anaheim, CA, 975598, 1–3. http://www2.isye.gatech.edu/~nemirovs%20/RO_LN.pdf http://www2.isye.gatech.edu/~nemirovs%20/RO_LN.pdf