Acta Polytechnica Vol. 43 No. 4/2009 Design Evaluation: Decomposition and State-space Analysis G. Green Thl Pa?er outLines the increasing d,emnnds upon eaaluation actiaity during the engineering design process. In partitular, the need to a!4.rex la.rge numbers of innoaattz,te concept options during the conceptual design phase is slressed and a six-step'method.ologl proposed. This methodoLogy combines and integratn techniques of inexact reasoning ,ih lt t need to cornbine two basic humnn alpioaihes to eaaLuation, namely fucornposition and holistic. The holutic eualuation eLem,ents cornprise fuzzl estimates of probability of achieiing pareto opthnal status cornbined with state+pace analysis. An exantple dentonstrates how these methids ,rny ,onotrge to pioaidc an aplropriate supportfor hurnan eualuation of emerging designs. It is concluded tlmt the six-step methodology exhibits aatidiiy and, time reductioi in terms of protiding an aid to the eaaluation of a large number of merging designs and their associated d.esign chaiacteristics. Keyuords: eualuation, decomposition, pareto optirnal, inexact reasoning, state-space. I Introduction The ability to rapidly evaluate design ideas is an essential element in the goal to increase design productivity. Given the need for companies to produce more innovative products in an increasingly competitive market place it follows that designers have to consider an increased number of design options if the most appropriate design is to be pursued through the product development process. Only through the generation of a relatively large number of concept design options along with a rapid and reliable means of evaluating the options will designers be able to increase design pro- ductivity whilst identi$ing and developing new innovative products. It is recognised that a significant difTicultyrvith eval- uating design options is that they are 'information poor'. That is, important decisions often have to be taken with very limited information It]. This provides designers rvith a major challenge and requires the provision ofdesign tools and aids. These are likely to be implemented within a computer-based environment [2]. It follows that the theoretical models under- pinning the design tools must be shown to be valid, reliable and robust [3]. Research activity being undertaken at the University of Glasgow is guided by a concurrent model of evaluation. It assumes two parallel strands, Holistic and Decomposirion. The Holistic approach takes a complete integrated view of the design artefact and seeks to provide an evaluation of the acceptability of it. The decomposition approach, on the other hand, evaluates the design at design characteristic level and then recombines these into an overall evaluation that can then be compared with the outcome of the Holistic approach. This approach is summarised and illustrated by a six-step methodology, shown in Fig. l. The main elements of the methodology are now briefly reviewed. These are: Decompo- sition of Design, Holistic Approach, Pareto Optimality. The objectives of this paper are: l. To outline the essential role of the evaluation activity throughout the product development process. 2. To report the current state of research activity aimed at understanding how design time can be significantly re- duced via support of evaluation activity at the 'fuzzy' front-end of the design process. 3. To specifically report the status of work aimed ar com- bining decomposition of design with space-state analysis to support human evaluation and selection of design concepts. 4. To summarise future research into design evaluation activity. This paper therefore describes a method that models the perceived dependencies between design criteria, whilst main- aining the benefits of decomposition. This method is then linked to an approach enabling the relative estimation of the probability that a design concept will ultimately meer the requirements of the declared design criteria. This linking is shown to provide an enhanced capability to support human evaluation and selection ofconcept designs. 2 Decomposition of design Decomposition of design is well established in practice as a means of trying to simplifi the complexity of design activity [4]. Indeed recent work has even reported on the strategic decomposition approach for conceptual design [5]. Decom- position involves trying to deal with the complexity of design by both describing the required design as a set ofcharacteris- tics and also by undertaking design synthesis of sub-functions rather than trying to adopt a holistic approach. The underly- ing assumption is that the resulting recomposed design will satisfy holistic evaluation. That is, there is an assumption of independence between the criteria. Howeve4 this is clearly not the case in most practical situations and what remains unclear is how the relationship between design characteristics should be modelled, to reflect their dependent nature, and how they may be recombined to provide a more realistic and holistic evaluation of the complete design. The models that seek to describe decomposition take as fundamental the idea of a sub-division of design space. That is, for a given design domain, the associated design characteristics (D"r) allow all associated specifications and models to be described in terms of the values of the characteristics. A Product Design Specifi- cation (PDS) can, in turn, be viewed as comprising a set ofD,r. The activity of evaluation then consists of the evaluator mak- ing ajudgement as to whether a particular design concept will meet the target value of each D.,, in turn. This can be illus- ll Acta Polytechnica Vol. 43 No. 4l\00g Define the Design Criteria and their Target Value or Value range. Estimate the value of the design characteristic for each concept. Judge the probabiliry that a Concept will achieve Pareto optimal status in next design phase. A state transition matrix, comprising 4 probabilities will result. Define the relation-sets within the design characteristics. Implied probability assigned to design characteristics. The matrix is multiplied by itselfto obtain a steady state probability that the concept will achieve Pareto optimal status.A relation-set probability and concept probability is evaluated. Compare the results of the decomposition approach with the holistic approach. t*'l t-*" l f-t*'l [""* ] t*' I t-t,"-f Fig. I : Summary of evaluation methodology trated most simply by the use of interacting uniform (recran- gular) distributions, as shown in Fig. 2. D,lValue (v) Fig. 2: Interaction of uniform distributions The degree of interaction and hence the degree of match between the desired target value range and the estimated tar- get range is given by the Design Margin (DM), as follows; t2 where, pDarT and pDrlrE are respectively the mean values and o7 and op 2r€ respectively the standard deviations of the Target and Estimation distributions. It has been shown [2] that the relationship between the design characteristics, when each is equally imporrant, can be modelled by: Note: n = the number of design characteristics. In situations where it is not essential that all design charac- teristic targets are achieved then one possible approach is to convert the DM's to an implied probability using the stand- ardised cumulative uniform distriburion as shown in Fig. 3. In this case the distribution has been standardised over a range of 0 to 4, thus if for example the DM has a value of 2 then the (l) (r\ Acta Polytechnica Vol. 43 No. 412002 implied probability @inpri.a) that the target will be achieved is 0.5. Fig. 3: Standardised cumulative uniform distribution The values on the x-axis represent the number of stan- dard deviations (z) of separation between a selected value (x) and the mean value (p,). Four standard deviations represent almost 100 7o of possible values. This value (z) is determined from the lollowing equation: v - tl z="- rx (3) Thus, from Fig. 3, if the value of z =0 then rhere is no separation between the mean and the selected value and therefore there is 100 % match and hence an implied proba- bility value of I is selected. Fig. 3 is analogous to a reliability curve. The underlying assumption here is that a'relation-set'can be defined that links a set ofdesign characrerisrics such that if at least one were to bejudged on-target then there is a possi- bility that the design concept may develop ro rowards a suc- cessful conclusion. This is, analogous to a cut-set in a complex reiiability network. A further assumprion is that each design characteristic is independent of the others. In some situations this assumption may not be valid and ther"efore musr be the subject of continuing research. In the meantime the following equation has been shown [3] to provide a valid method to aid identification and selection of appropriate design conceprs. D/ - )!+l #Vl I l/, , \l /,tr 1(l-li*prr.a.)l (4) -.LL t=l \ i=l ) Note: n = the number of design characteristics within a de- fined relation-set R = the number of relation-sets Furthe4 the relative importance of each design character- istic, usually defined within traditional evaluation merhods by an individual weighting, can be seen to be inappropriate when the interaction of design characteristics takes place. A more acceptable and logical approach is to define the im- portance or criticality of each characteristic in terms of the desired degree of match with the design specificarion rarger levels. 3 Holistic evaluation It is human nature to try to reduce complexity when at all possible, The aim is to obtain a usable answer as quickly as possible. Designers are no exception to this trait and have therefore traditionally relied upon using experience to rnake holisticjudgemenrs abour design concepts. That is theyjudge the overall acceptability ofa design concept rarher rhan using methods of decomposition. This may be acceptable if the evaiuator's experience is completely appropriate, but what happens to the reliabiliry of the merhod when faced with new or innovative concepts? To understand this point further ir is necessary to consider how a design concept progresses through the design process. We can say that a design changes state, from being an idea to a concept to an embodied design through to a detail design capable of manufacture, as it passes through the design process. So any judgement of a concept requires consideration or prediction of whether the said concept will satisfactorily progress through the process or whether it will fail to meet the demands of the design specifi- cation at some point. -fhis is effectively the holisticjudgement that is being made. State-space, in a design context, is con- cerned with analysing the probability of furure srares of a system, or design, given knowledge of the probability of the system moving benveen states [6], in our case during the design process. In the conrext ofdesign this involves evaluat- ing the probability that a concept will meet the inter-related rtquirements of a design specification and thus be judged to be in a 'working' state. As the concept progresses through phases of the design process the probability of being in a working state will vary as information becomes richer. A view of the likliehood of a particular concepr conrinuing to be in a 'working' state at the end of the design process will go a longway to supporting the initial concepr evaluation and selection decision. An acceptable or a working state can, in this context, be defined as Pareto Optimal. That is,,4 conceptis consilercd Pareto optirnnl if in atternpting to ntoue the aalue of a par- ticular desi,gn characteristic closer to its target aalue or ronge the effect is to rttoue another of the duign criterin away from its target ualue or acceptable range'. In effect a Pareto Optimal state is one where you cannot make an improvement in one design characteris- tic without having a negative effect on another. In making holistic judgement an evaluator is effectively having to make a subjective (fuzzy) estimate [7, 8] of the follorving: l. Probability of maintaining a Pareto optimal srate. 2. Probabiliry of moving out with a Pareto optimal srare. 3. Probability of remaining out with a Pareto optimal srare. 4. Probabiliry of moving into a Pareto optimal state. A human evaluator makes a holisticjudgement, using a 0 to I scale, for each concept under evaluation. For example, let us assume that the estimated probabilities are as follows: 0.9, 0.1, 0.6, 0.4. This would allow us to construct aFvzzy Tiansi- tion Probability Matrix which when multiplied by itself a sufficient number of times will reach a steady-state condition for the concept indicating its overall likliehood of being in a Pareto optimal state at the end of the process. i-0.9 0.1l " Lon 06l (5) In this case a steady state is reached after approximately seven intervals (n), this is illustrated in Fig. 4. >' * n<5 "'" v l3 Acta Polytechnica Vol. 43 No. 4/2003 \-- \ ---+-__+- 4 State lntervals Fig. 4: Steady-state probability Table l: Summary of method for determination of concept scores Dch Implied Probability Relation Concept Set Score A B C D E 0.78 0.80 t.77 0.55 0.78 0.80 t.74 0.56 2.50 0.38 l.ct Dch Implied Relation Probability Set A B C D E 1.74 0.56 0.00 r.00 t.74 0.56 0.62 0.84 2.50 0.37 6.61 Dch A B c D E Design Implied Relation Margin Probability Set 1.74 0.56 1.00 0.75 3.48 0.13 0.52 0.87 t.77 0.55 DchT J t4 Acta Polytechnica Vol. 43 No. 4/2003 'The holistic approach provides a subjective probability that the concepr being evaluated will remain pareto optimal throughout the design process. In the above case a subjective probability of 0.8 is suggested. If this figure exceeds rhar asso- ciated with other concepts, rhen this concepr would be a strong contender for selection. Of course the methodology described in this paper requires that the resuit obtained via holistic evaluation be compared with rhat produced via the decomposition approach before a final conclusion is reached. A simple example will now be used to illustrate this comparison. 4 Example The fbllowing simple example is only designed to illus- trate the methodology in action and does not represent a real situation. Let us therefore assume that rve have produced 3 concepts as potential design solutions to a particular prob- lem. Let us further assume that each concept is defined bv 5 design characteristics (A-E). Step I - In accordance with the methodology rve firsr have to define design characterisric target (D.17) values. These will obviously be the same for each concepr. Step 2 - The evaluator estimates the likely value of the De- sign Characteristic (D.7,.8). Step 3 - ll.elation sets are determined for the Design Char- acteristics. In this example we assume that two relation sets exist, namely: {A, B, C}, {D, E}. Step 4 and 5 - The Design Margin is evaluated and the implied probabilities gained from the interacrion DrlT and Dr1,E are used in conjuncrion rvith equa- tion 4 to obtain a measure (Concept Score) of the concept that best meets the requirements. The results of the above are summarised in Thble l. From the results in Thble I we can see that the summation of Design Margins for each concept indicates that Concept 2 has the best overall match, with a score of 6.61. Howeve4 when we seek to take the relationship between design charac- teristics into account by using the implied probability ro evaluate the relation seti we find that Concept 2 remains the best choice but that the order of the other two concepts has changed. Note that with the Design Margin approach, rhe smallest score is best, and with the relation set approach, the biggest score is best. Step 6 - Requires comparison between the above and the holistic judgement using a state-space approach. Let us again assume that the evaluator's judge- ments about the three concepts are as summarised in Table 2. The resultant steady-state probability of the concept re- maining in Pareto optimal state is determined using the method outlined earlier in the text. We see once again that concept 2 is identified as the best ofthe three being evaluared. Our next step is to compare this result with those attained via the decomposition approach incorporating relation sets. To ease comparison the results are summarised in Thble 3. In the above example we see that Concept 2 has main- tained dominance of the three options but that the ranking has changed from 2,1,3 with Design Margin, ro2,3,1with the Table 2: Summary of evaluatorjudgements Table 3: Compalison of evaluation results Decomposition Approach Holistic Approach Design Margin Relation Set Concept I t.o 0.71 0.80 Concept 2 6.6 0.90 0.86 Concept 3 d.J 0.88 0.77 Relation set approach and then back again to 2, 1,3 r,vith the holistic approach. 5 Conclusions This paper has outlined the increasing demands upon evaluation activity during the engineering design process. In particular, the need to address large numbers of innovative concept options during the conceptual design phase has been strcssed and a six-step methodology proposed. This method- ology has been shown to comprise techniques of inexact reasoning with the need to combine two basic human ap- proaches to evaluation, namely decomposition and holistic. It is clear to the author that the methods employed are only useful if they can be implemented within a computer environ- ment, otherwise engineers and designers are likely to fall back on traditional approaches rather than the detailed approach of the six-step methodology. Only through computer imple- mentation will the desired time reduction be achieved rvhilst being able to process data associated with a large number of design concepts and their associated set ofdesign characteris- tics. This is work that is on-going within the Department of Mechanical Engineering at the University of Glasgow. A par- ticular aspect of this research is the need to better model the interaction and interdependency of design characteristics and to further test the six-step methodology in active indus- trial environments. Prrcbability Esrimares Concept I Concept 2 Concept 3 Probability of maintaining a Pareto optimal state 0.9 0.95 0.85 Probability of moving out with a Pareto optimal state 0.t 0.05 0.2 Probability of remaining out rvith a Pareto optimal state 0.6 0.3 0.5 Probability of moving into a Pareto optimal state 0.4 0.7 tr.3 Resultant Steady-state Probability 0.80 0.86 0.77 lc Acta Polytechnica Vol. 43 No. 412009 References tll Wu, H., Wang, F.; Zhang, X., He, N.: A Modtl of Inexact Reasoning in Mechnnirnl Design Eaaltntion. Artificial Intel- ligence in Engineering, Vol. 10, 1996, p. Zb7-262. t2l Green, C.: Modzlling Concept Design Eaahtatian. Artifuial Intelligence for Engiruering Desr.gz. Analysis and Manufac- ture (AIEDAM), Vol. I I, 1997, p.2lt-217. t3l Green, G.: Towards Integrated Daign Etalmtinn: Valida- ti.on of Mod.ek Journal of Engineering Design, Vol. Il, No.2,2000, p. l2l-132. Ho, C. H.: Somc Phmom.ena of Problem Decomposition Strat- egy for Design Thinking: Dffermces between Noaices and. Ex- perts. Design Studies, Yol 22, No. l, 200.1, p.27-4b. Medland, A. J., Muillineux, G.: A Decomposition Strategy for Cancephnl Daign. Journal of Engineering Design, Vol. ll, No, 1,2000, p.3-16. Zhang, W. Y., Tor, S. B., Britton, G. A.: A Heuristic Stnte+pace Approach to thc Functiona.l Duign of Mecfu,nicat Slstens. International Journal of Advanced Manufac- turing Technology, Vol. 19, No. 4,2002, p.2Zb-244. Wang, J.: A Funy Ouhanhing Method for Cmceptrnl De- sign Evaluntion InternationalJournal of production Re- search, Vol. 35, No. 4, 1997, p. 995-1010. Vanegas, L. V., Labib, A. W.: Application of Neu Fuzzl-weighted Auerage (NWA) Method to Engineering De- sign Eualtntion International Journal of production Research, Vol. 39, No. 6, 200 I , p. I I 47- I I 62. 171 t8l t4l t5l i6l Dr. Graham Green e-mail: g.green@mech. gla.ac.uk Department of Mechanical Engineering James Watt Building University of Glasgow Glasgow cl2 8QQ Scotland, United Kingdom l6 Scan1 11 Scan1 12 Scan1 13 Scan1 14 Scan1 15 Scan1 16