AP05_3.vp 1 Introduction In engineering design a designer needs to satisfy a set of functional requirements within a given set of constraints. However, a good engineering design is one that goes beyond merely satisfying these requirements within constraints but achieves a certain level of excellence in some quantifiable or unquantifiable manner. Put in another way, designers seek to optimise designs during the design process. Design optimisa- tion often involves conflicting multiple objectives or criteria which can be regarded as a form of multiple criteria decision making [1]. Multiple Criteria Decision Making (MCDM) can broadly be classified as: � Synthesising a set of competing design alternatives. � Selecting the most preferred design(s) from a set of competing design alternatives. The search for optimum design solutions involving multi- ple objectives during the synthesis process usually results in non-dominated or efficient solutions. The search for an efficient solution begins in the feasible solution space and a bi-objective solution design optimisation solution is shown in Fig. 1. Criterion 1 and criterion 2 are to be maximised, and points A and B are the optimum design solutions if criterion 1 and 2 are optimised as two single objec- tive optimisation problems. The unattainable ideal solution is represented by point “O” in Fig. 1. It is clear, from Fig. 1, that all the design solutions in the shaded region dominate solu- tion “X”. If a solution on the Pareto surface is found, then it is usually sensible to take it to represent the “best solution” in that no improvements can be made on either criterion with- out sacrificing the performance of the other criterion. This realistic approach of incorporating conflicting objectives in a optimisation framework finds readily available applications in various fields of engineering design: for example safety de- sign [2] and finite element analysis during design [3]. Various methods have been developed that allow one to search for solutions on the Pareto surface Two such methods are the Interactive step trade-off method and the multiple objective genetic algorithm [1, 4]. The selection of the most preferred design solution(s) from a set of efficient design solutions is a subjective matter and depends on the decision maker’s preference. In general, given a set of design alternatives, the decision maker then analyses the merits of the various attributes (e.g. cost, perfor- mance, and appearance) on the basis of preference structure before ranking or selecting the most preferred design alterna- tive(s). Again, various methods have been developed to allow one to rank and select the most preferred design alternatives from a given set of alternatives and articulation of the de- signer’s preferences [5, 6, 7]. 2 Preference structure In MCDM, it is a difficult task to elicit a designer’s or deci- sion maker’s preference structure. The preference structure of the designer or decision maker is usually expressed through weights or utility functions. The preference structure may be elicited in terms of pairwise comparison of attributes (or crite- © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ 5 Czech Technical University in Prague Acta Polytechnica Vol. 45 No. 3/2005 Elicitation of Preference Structure in Engineering Design J. K. Tan Engineering design processes, which inherently involve multiple, often conflicting criteria, can be broadly classified into synthesis and analysis processes. Multiple Criteria Decision Making addresses synthesis and analysis processes through multiple objective optimisation to generate sets of efficient design solutions (i.e. on Pareto surfaces) and multiple attribute decision making to analyse and select the most preferred design solution(s). MCDM, therefore, has been widely used in all fields of engineering design; for example it has been applied to such diverse areas as naval battle ships criteria analysis/selection and product appearance design. Given a list of design alternatives with multiple conflicting criteria, preferences often determine the final selection of a particular set of design alternative(s). Preferences may also be used to drive the design/design optimisation processes. Various methods have been proposed to model preference structure, for example simple weights, multiple attribute utility theory, pairwise comparison, etc. Preference structure is often non-linear, discontinuous and complex. An Artificial Neural Network (ANN) learning-based preference elicitation method is presented in this paper. ANNs efficiently model the non-linearity, complexity and discontinuity nature of any given preference structure. A case study is presented to illustrate the learning-based approach to preference structure elicitation. Keywords: engineering design, multiple criteria decision making, preference structure. Feasible region Infeasible region Pareto surface Fig. 1: Solution space of a bi-objective design optimisation problem ria), ranking of all attributes, ranking of a sub-set of alterna- tives with respect to all attributes, and the definition of ideal and negative ideal solutions. Given the fact that the decision maker may not be able to articulate the preference structure through the comparison of pairs of attributes and/or solu- tions, and that comparison of pairs of attributes may not be adequate to capture the interactions between the attributes of a decision making problem, the results of preference elicita- tion may not be well-agreed by the decision maker. In general, the greater the volume of preference information that is provided, the higher is the accuracy of the weights or utility function obtained, accompanied by a higher risk if inconsis- tencies in judgement are manifested during the elicitation process. Attempts are being made to take the complexity of preference elicitation into account in MCDM. One such example involves the use of Artificial Neural Networks and fuzzy set theory to model preference relations for MCDM [8]. Artificial Neural Networks (ANNs) have been used in a large range of applications in many fields [9]. ANNs are par- ticularly good at recognising complex patterns and images when they are appropriately set up and trained. One such example involved the use of ANN to map the complex re- sponse surface of hydrodynamic performance [10]. For the difficult task of eliciting a decision maker’s preference, a learning-based approach using ANN is proposed in this pa- per to capture the designer’s or decision maker’s preference structure. The proposed learning-based approach is an itera- tive one that allows the designer or decision maker to state and refine his preference on a set of competing design alter- natives. This work is still in its early stage and hence only the results of a preliminary investigation are illustrated as a simple case study example. Further results will be dissemi- nated in due course as work progresses. 3 An example The preliminary investigation of this proposed learn- ing-based approach has been carried out using a set of existing data on a catamaran design problem. The efficient design solutions data for this catamaran design problem is adapted from the example of [1], which uses a utility function to capture the preference structure, and the learning-based approach is applied to illustrate the decision maker’s prefer- ence. In this problem, a catamaran vessel is designed by modifying a parent catamaran hull form so as to maximise a number of performance measures. These performance mea- sures (termed attributes, objectives or criteria) are heave, pitch, roll and relative bow motion of the vessel. To optimise the performance for robustness over a range of wave head- ings, the signal-to-noise ratio is maximised. The results are shown in Table 1. The non-dominated optimal or efficient solutions on the Pareto surface are obtained (see Table 1) by creating vari- ant designs through a simple perturbation of the three primary design variables, these being the length (L), the beam over draft ratio (B/T) and separation of the demi-hulls 6 © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ Acta Polytechnica Vol. 45 No. 3/2005 Czech Technical University in Prague Parameters Criteria �L % �B/T % �HS % Heave(dB) Pitch(dB) Roll(dB) RBM(dB) 1 �5.0000 �10.0000 �10.0000 6.9372 15.3381 2.4447 6.3923 2 5.0000 10.0000 10.0000 6.8438 19.1872 2.5865 6.8327 3 0.0000 10.0000 10.0000 6.1819 7.9100 �0.3776 8.5016 4 5.0000 5.0000 10.0000 6.9329 16.7476 7.4053 6.4656 5 0.0000 5.0000 10.0000 6.1645 7.8047 5.5222 10.0961 6 10.0000 0.0000 10.0000 6.8808 13.0745 10.8571 5.2839 7 5.0000 0.0000 10.0000 6.9001 11.8422 8.1982 5.0751 8 �5.0000 �5.0000 0.0000 6.9762 13.4082 5.4809 4.9013 9 �10.0000 �5.0000 0.0000 7.2764 11.1131 4.7353 4.3594 10 0.0000 �5.0000 �10.0000 6.1508 7.6588 8.3186 9.8139 11 �10.0000 �5.0000 �10.0000 7.2764 11.1131 3.8911 4.9367 12 �5.0000 �5.0000 �10.0000 6.9762 13.4082 4.7291 5.6957 13 �10.0000 �10.0000 �10.0000 7.2411 12.9420 3.6884 5.7528 14 10.0000 10.0000 10.0000 6.8514 17.9598 6.9071 6.1825 15 10.0000 5.0000 10.0000 6.8908 15.8289 9.1926 5.9075 16 �10.0000 �5.0000 10.0000 7.2764 11.1131 5.0582 3.7533 17 10.0000 �5.0000 10.0000 6.6281 9.9164 11.4248 3.8082 18 10.0000 �10.0000 0.0000 7.2411 12.9420 3.9183 5.2209 Table 1: Efficient solutions for the example problem (HS). The design variations for the three primary variables are as follows: L :1 � 0.1 in steps of 0.05 (i.e. � 10 % variation in steps of � 5 %) B/T : 1 � 0.1 in steps of 0.05 H : 1 � 0.1 in steps of 0.05 It is obvious from the data that it is not possible to maxi- mise all four criteria simultaneously, and a trade-off between the four criteria will be necessary. The designer or decision maker needs to articulate his preference so as to identify the “best” design. More importantly, due to the nature of the problem, the 18 design alternatives presented may not contain the most desirable features in accordance with the yet-to-be captured preference structure. Hence the prefer- ence structure can be used to guide a further explorative search in an attempt to cover improved designs. To aid the decision maker in the articulation, an overall scoring system between 0 and 1 will be used to rank the design alternatives. A feed-forward ANN, as shown in Fig. 2, is set up to map and capture the preference structure. The input nodes i1– i4 are used to receive the 4 performance measures of heave, pitch, roll and RBM, and the output node o1 will be used to receive the decision maker’s overall preference. The learning-based approach allows the decision maker or designer to articulate his preference in an incremental manner: � The decision maker first selected four design solutions to represent the best, good, average and poor designs with appropriate scores. � The ANN was then trained to capture this initial prefer- ence structure. This preference structure is then used to predict the scores of other solutions. � If the decision maker agreed with the predicted scores then those scores would be used as additional training data. Otherwise, the decision maker assigned new scores and these new scores are then again used as additional training data set. � The process is then refined/repeated until the decision maker is satisfied that the preference structure had been © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ 7 Czech Technical University in Prague Acta Polytechnica Vol. 45 No. 3/2005 Output layerInput layer First hidden layer i1 i2 i3 i4 o1 Fig. 2: ANN set up for the example Fig. 3: Correlation of decision score and performance measures mapped adequately. This process can be further refined at a later stage if necessary. The influence of the decision maker’s preference on indi- vidual performance measures over the final score can be revealed through simple scatter plots, as shown in Fig. 3. It can be seen from Fig. 3 that the decision maker, unconsciously perhaps, did not consider heave and roll to be equally impor- tant considerations as pitch and RBM affecting the overall selection of final design solution. Pitch and RBM thus became important in the selection of the final “best” design solution. The influence of pitch and RBM on the score can thus be plot- ted (shown in Fig. 4) to reveal the preference structure and the interrelation of these two criteria in relation to the final score (desirability of the decision maker). Intuitively, one would sus- pect a correlation between RBM and pitch, and indeed Fig. 4 shows that there is a good correlation between the two impor- tant performance measures of RBM and pitch. From the graphs shown in Figs. 3 and 4, the decision maker decided to look more closely at the two criteria pitch and RBM. An analysis of these two criteria on the basis of the given 18 competing designs yielded the graph shown in Fig- ure 5 which revealed that the influence of RBM somehow peaked at the value of 6, whereas pitch has a stronger influ- ence, in that a higher value of this criterion is always desirable. The observation derived from this simple exercise is not dissimilar to those obtained from [1], using the utility func- tion approach, in which the decision maker used a pair-wise comparison of competing designs to construct the utility functions of the performance measures: pitch, RBM, heave and roll. It should be pointed out that due to the difference in the nature of the evaluation algorithm, the results in terms of numerical values obtained by this method should not be com- pared directly to those stated in [1], which was computed us- ing utility functions. However, the overall trend of the prefer- ence structure should not be dissimilar between the two meth- ods. The resultant utility functions derived from the analysis are linear for pitchThe higher the values of the pitch signal to noise performance, the higher will be the utility values and hence the desirability. The RBM’s utility curve showed signifi- cantly less influence than the pitch’s utility curve, and the val- ues of the RBM utility were virtually constant and at a value significantly lower than those associated with pitch. The preference structure elicited is then used to assist the designer to perform a further explorative search in an at- tempt to obtain better overall desirability in accordance with this preference structure. Clearly, the direction of the search is directed at trading off the performances of heave, roll and to some extent RBM in order to gain performance in terms of pitch performance. Again, a similar conclusion has been drawn in [1]. A further search for solutions can indeed be per- formed on the basis of this preference structure, but for brev- ity the details of this are not presented here. 4 Discussion and conclusion Elicitation of a preference structure is not an easy task, principally because it involves articulation of human prefer- ence over a set of competing alternatives. The subjective nature of the design selection process, which involves multi- ple conflicting criteria, requires trading-off between attributes or criteria with possible interactions between these attrib- utes or criteria. Various methods have been developed to assist decision makers in the articulation and mapping of an underlying preference structure. This paper presents a learn- ing-based iterative approach that allows the decision maker to form the preference structure incrementally by stating his preference using a simple scale system. Admittedly, one cannot claim that this learning-based iterative approach is perfect; however the decision maker can, through a series of intuitive refinements, arrive at some credible preference structure on the basis of his intuitive articulation. The artifi- 8 © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ Acta Polytechnica Vol. 45 No. 3/2005 Czech Technical University in Prague Pitch RBM Score Fig. 4: Plots of preference structure & pitch vs RBM Fig. 5: Pitch and RBM vs score cial neural network handles the complexity of the possible in- teractions of the criteria through learning efficiently and transparently via examples and training so as to map the com- plex response surface that fits the given training data. The preference structure derived can be used to perform further explorations of the solution space in search of better overall desirable design performance. A quantitative example is presented to illustrate the use of this approach and the resultant preference structure. How- ever in many design problems, certain attributes or criteria are somewhat unquantifiable (e.g., the shape and appearance of a design, which often significantly affects the product’s desirability), and eliciting the preference structure of a deci- sion making problem involving unquantifiable attributes and criteria poses a significant challenge to researchers and prac- titioners in the field of decision making and design. The work described in this paper is still in its early stage and therefore has not specifically addressed this issue in depth; however, it is noted that shape factors (an important attribute for industrial designers) may be correlated to customer preference [11]. It is noted, therefore, that the proposed approach can potentially be gainfully employed over a wide range of applications in multiple criteria decision making. In conclusion, the learning-based approach, on the basis of this preliminary investigation, appears to be intuitive and attractive and can potentially be used in a wide range of appli- cations including engineering design, industrial design and product design. References [1] Sen, P., Tan, K.: Multiple Criteria Issues in Design: Seeking the Right Balance, International Workshop on Multi-Cri- teria Evaluation, MCE 2000, Neukirchen, September 14th–15th, 2000. [2] Sen, P., Tan, L., Spencer, D.: “An Integrated Probabilis- tic Risk Analysis Decision Support Methodology for Systems with Multiple State Variables.” Reliability Engi- neering & System Safety, Vol. 64 (1999), No. 1, p. 73–87. [3] Zhang, W. H.: “Pareto Optimum Sensitivity Analysis in Multicriteria Optimization.” Finite Elements in Analysis and Design, Vol. 39 (2003), p. 505–520. [4] Pereira, C. M. N. A.: “Evolutionary Multicriteria Optimi- zation in Core Designs: Basic Investigations and Case Study.” Annals of Nuclear Energy, Vol. 31 (2004), p. 1251–1264. [5] Jacquet-Lagreze, E., Siskos, J.: “Assessing a Set of Addi- tive Utility Functions for Multicriteria Decision Making, the UTA Method.” European Journal of Operation Re- search, Vol. 10 (1982), p. 151–164. [6] Hwang, C. L., Yoon, K.: Multiple Attribute Decision Making – Methods and Applications: A State-of-the-art Survey, 1981, Springer-Verlag, Berlin. [7] Saaty, T. L.: “How to Make a Decision: the Analytical Hierarchy Process.” Interfaces, Vol. 24 (1994), No. 6, p. 19–43. [8] Wang, J.: “A Neural Network Approach to Modelling Fuzzy Preference Relations for Multicriteria Decision Making.” Computers and Operations Research, Vol. 21 (1994), No. 9, p. 991–1000. [9] Patternson, D. W.: Artificial Neural Networks: Theory and Application, Eaglewood Cliffs, NJ Prentice Hall, 1997. [10] Tan, K., Sen, P.: “The Application of a Decomposition and Reuse Approach in Marine Design.” PRADS 2001, Practical Design of Ships & Other Floating Structures, 8th Intl Conf, September 16th–21st 2001, Shanghai, China. [11] Vergeest J. S. M., Egmond, R., Dumitrescu, R.: Correlat- ing Shape Parameters to Customer Preference, Proceedings of the TMCE, 2002 April, Wuhan, China. John K. Tan, Ph.D. e-mail: k.tan@unn.ac.uk School of Engineering & Technology Ellison Building Northumbria University Newcastle upon Tyne, NE1 8ST United Kingdom © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ 9 Czech Technical University in Prague Acta Polytechnica Vol. 45 No. 3/2005