Plane Thermoelastic Waves in Infinite Half-Space Caused Decision Making: Applications in Management and Engineering Vol. 4, Issue 2, 2021, pp. 1-25. ISSN: 2560-6018 eISSN: 2620-0104 DOI: https://doi.org/10.31181/dmame210402001a * Corresponding author. E-mail addresses: ahmataytekin@artvin.edu.tr (A. Aytekin) COMPARATIVE ANALYSIS OF NORMALIZATION TECHNIQUES IN THE CONTEXT OF MCDM PROBLEMS Ahmet Aytekin1* 1 Department of quantitative methods, Artvin Çoruh University, Turkey Received: 30 January 2021; Accepted: 3 March 2021; Available online: 21 March 2021. Original scientific paper Abstract: Normalization is an essential step in data analysis and for MCDM methods. This study aims to outline the positive and negative features of the normalization techniques that can be used in MCDM problems. In order to compare the different normalization techniques, fourteen sets representing different scenarios of decision problems were used. According to the results, if the decision-maker chooses to take the alternative with the highest value in the criteria and avoid the one with the lowest value, or vice versa, optimization-based normalization techniques should be preferred, whereas the reference-based normalization techniques are considered appropriate for situations where there are ideal values determined by the decision-maker for each criterion. However, if the decision-maker believes that the values in the criteria do not represent the monotonous increasing or decreasing benefit/cost, then non-linear normalization techniques should be used. Also, in the event of a change in the conditions mentioned above, the decision maker may opt for mixed normalization techniques. However, some data structures, such as the presence of zero, and negative values in the decision matrix, can prevent the use of some normalization techniques. The choice of the normalization technique may also be affected by the problem of rank reversal, the range of normalized values, obtaining the same optimization aspect for all criteria, and the validity of results. Key words: Data; MCDM; Scaling; Normalization. 1. Introduction In quantitative research, researchers often try to use methods appropriate for the data structures. To do this, the data is first collected and then compiled. In the compilation process, it is always important to create the data structure required by the relevant method using scaling techniques. In the scaling process, the unit of measurement, the size, and the level of the criteria are changed alongside one or more of the transformations, re-measurement, mailto:ahmataytekin@artvin.edu.tr Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 2 normalization, or weighting operations. The differences in various features of the criteria such as measurement levels, size of the range, importance levels and reflecting the decision maker's preferences effectively are prominent reasons for scaling. The other reason for scaling is the need to meet the assumptions of the method used for the research or the decision problem. In this context, the primary purpose of scaling is to provide the appropriate measurements or data structure for the proper method or analysis. Normalization is one of the critical processes used in scaling data (Jensen, 1984; Roberts, 1984; Lootsma, 1999; Tavşancıl, 2006; Kainulainen et al., 2009; Sarraf et al., 2013; Jahan & Edwards, 2015; Podviezko & Podvezko, 2015; Gardziejczyk & Zabicki, 2017). There are numerous application areas for normalization including data mining, multivariate statistics and multi-criteria decision making (MCDM) among others. This study will however focus on the effects of the normalization processes on solutions obtained using MCDM methods. Normalization is used to obtain criteria that have the same weight, are dimensionless, and are suitable for compensatory processes in MCDM problems. Normalization also enables the decision maker to show his preferences regarding the problem to a certain extent. There are many normalization techniques in the literature to achieve this. The choice of the normalization technique depends on the structure of problems and the assumptions of MCDM methods. Although not yet sufficient, studies on the comparison of normalization techniques have been increasing in the past decades. These studies, however, usually include a small number of techniques. Similarly, studies considering the selection of the normalization technique, and the criteria to be used in this selection process are also limited. Another important issue is that every normalization technique cannot be suitable for all decision problems. It is, therefore, necessary to investigate the extent to which the normalization techniques have achieved their purposes of development, their roles in the problem, and the MCDM method, their dimensionlessness, and comparability. In this context, this study will examine the practical comparisons of the normalization techniques, determine the positive and negative features, and outline the selection process of normalization techniques suitable for different data structures. The purpose of the study is thus to provide different perspectives on normalization techniques and a holistic framework for researchers and decision makers. 2. Normalization Normalization is a scaling process used to make the criteria comparable by eliminating the optimization orientation (benefit, cost), the unit of measurement, and the variation range. Through normalization, the data is converted to a specific norm or standard. Another term often used interchangeably with normalization is standardization. However, standardization is a normalization process that eliminates unit differences and transforms values to a specific range, such as 0-1 in all criteria. In general, normalization techniques are expected to equalize the effect levels of all criteria (regardless of the weighting process), process the zero and negative values, generate the same normalized value for different units of measure that can be converted into each other (as in the case of g / cm3 – kg / m3), and not cause rank reversal problems while also ensuring symmetry in the cost and benefit optimization orientation. The normalization technique that has these features is considered successful (Pavličić, 2001; Jahan & Edwards, 2015; Podviezko & Podvezko, 2015). Comparative analysis of normalization techniques in the context of MCM problems 3 In the analysis of normalization techniques in the MCDM literature, the decision matrix given in equation (1) will be used. The rows of the decision matrix contain alternatives while the columns carry the criteria. Each of its cells/elements shows the quality, feature, or performance value of each alternative in the relevant criterion.                       11 1 1 1, , 1, , n m mn i mx x j nX x x (1) The elements of the decision matrix in equation (1) are expressed as xij. xij is the performance or result value of alternative i in criteria j. The following section of the study outlines the classifications of normalization techniques. 2.1. Classification of Normalization Techniques Classification of normalization techniques makes it easier to identify the similarities and differences of the techniques, standardize the concepts in the field, and examine the increasing number of techniques. Various approaches can be used in the classification of normalization techniques. However, the most common classifications in the literature are done according to the distance measurements, the linearity of the normalization process or the optimization orientation of the criteria (Milani et al., 2005; Yoon & Kim, 1989; Zeng et al., 2013; Jahan & Edwards, 2015). Distance measurements are the most commonly used in the normalization process. The distance-based normalization is the ratio of the distances of the alternatives from the starting point (vector 0) to the sum of distances of all alternatives from the starting point in the relevant criterion. In the distance-based normalization processes, Eq. (2) is used in the Lp metric. (Yoon & Kim, 1989, p. 22):   1 1 0 ( ) 0 ij ij m pp ij i x n p x           (2) Eq. (2) is used in the benefit criteria. For the cost criteria, the values are converted to benefit with the transformation of 1/xij values. In Eq. (2), Manhattan distance normalization is performed for p = 1, with Euclidean distance normalization for p = 2 and Tchebycheff distance normalization for p = ∞ (Yoon & Kim, 1989). In the literature, Manhattan distance normalization is called Sum-Based Linear Normalization, while Euclidean distance normalization is known as Vector Normalization. For, normalization processes that are not based on distance, a specific value is used. Often, these values are the maximum and the minimum in the criterion. Similarly, reference values or large fixed numbers can be used. The linearity of the normalization process is that the utilities or values in the criterion increase or decrease monotonously in a specific direction. In non-monotonic normalization processes, there is no continuous increase or decrease of acceptable performance values within the criterion in a certain direction. For example, in the criterion with a normal distribution, the linear normalization process will not be Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 4 appropriate if the desired/ideal values are within three or four standard deviations of the mean. Z score normalization, some reference-based normalizations, and non- linear normalizations are some examples of non-monotonic normalization techniques (Zavadskas & Turskis, 2008; Zeng et al. 2013). We can divide normalization techniques into two fundamental classes based on whether they consider the optimization orientation of the criteria. However, some normalization techniques provide a mixed/integrated normalization process with the idea that the optimization orientation and reference value are vital for different criteria that can be found at the same time in the decision problem. The following section will examine the normalization techniques which depend on the optimization orientation, those that are independent of the optimization orientation, and those that have a mixed structure. 2.2. Normalization Techniques Depending on the Optimization Orientation Most normalization techniques provide normalization according to the optimization orientation of the criteria. The optimization orientation is divided into two - benefit and cost. The benefit optimization orientation implies that the increase in the performance values of the alternatives evaluated in criterion j is preferred to the decrease. The cost optimization orientation is that the decline in the performance values of the alternatives in criterion j is preferred to the increase. In general, we can say that the highest (maximum) value in benefit- orientation criteria and the lowest (minimum) value in cost- orientation criteria are preferred. Normalization techniques depending on the optimization orientation are given in Table 1 (Brauers & Zavadskas, 2006; Zavadskas & Turskis, 2008; Fayazbakhsh et al., 2009; Jahan & Edwards, 2015; Gardziejczyk & Zabicki, 2017). These techniques mainly use performance value totals, the maximum value, and the minimum value in a criterion. Table 1. Normalization techniques depending on the optimization orientation Notation Techniques Benefit Criteria Cost Criteria References N1 Sum-Based Linear Normalization    1 ij ij m ij i x n x    1 1 1 ij ij m i ij x n x Gardziejczyk & Zabicki (2017) N2 Vector Normalization    2 1 ij ij m ij i x n x     2 1 1 ij ij m ij i x n x Zavadskas & Turskis (2008) Gardziejczyk & Zabicki (2017) N3 Logarithmic Normalization          1 ln ln ij ij m ij i x n x            1 ln 1 ln 1 ij m ij i ij x x n m Zavadskas & Turskis (2008) Gardziejczyk & Zabicki (2017) N4 Maximum - Linear Normalization  ij ij ij i x n maks x  1 ij ij ij i x n maks x Jahan & Edwards (2015) Comparative analysis of normalization techniques in the context of MCM problems 5 Notation Techniques Benefit Criteria Cost Criteria References N5 Minimum - Linear Normalization   min 1 iji ij ij x n x  min iji ij ij x n x Jahan & Edwards (2015) N6 0-1 Interval Normalization Using Max- Min  ij ij ij i x n maks x  min iji ij ij x n x Brauers & Zavadskas (2006); Jahan & Edwards (2015) N7 Jüttler-Körth Normalization    1 ij ij i ij ij i maks x x n maks x    min 1 ij iji ij ij i x x n maks x Brauers & Zavadskas, (2006); Zavadskas & Turskis (2008); Gardziejczyk & Zabicki (2017) N8 Stopp Normalization  100 ij ij ij i x n maks x  100min iji ij ij x n x Brauers & Zavadskas (2006); Gardziejczyk & Zabicki (2017) N9 Nonlinear (Peldschus) Normalization         2 ij ij ij i x n maks x         3 ij ij ij i x n maks x Brauers & Zavadskas (2006); Zavadskas & Turskis (2008); Gardziejczyk & Zabicki (2017) N10 Weitendorf’s Linear Normalization    min min ij iji ij ij ijii x x n maks x x    min ij ij i ij ij ijii maks x x n maks x x Brauers & Zavadskas (2006) N11 Z Score Normalization Depending Optimization Aspect              1 2 1 m ij i ij ij j ij m j ij j i x xx mn x m      ij j ij j x n Fayazbakhsh et al. (2009); Gardziejczyk & Zabicki (2017) It is possible to further divide the normalization techniques depending on the optimization orientation into four sub-classes: sum-based, a maximum or minimum value-based, range-based, and others. From the techniques in Table 1, while N1, N2, and N3 are sum-based, N4-N9 are maximum-minimum value-based, N10 is range- based, and N11 is evaluated under the other category. In sum-based normalization techniques, the sum of performance values within the criterion is used. It seems that the normalization techniques in this class may lead to the rank reversals problem due to the changes (adding or removing alternatives) in the alternative set. For example, when the alternative performance with the highest performance value is removed from criterion j, which has a benefit optimization orientation, the maximum value used in N4 will change. Change of the maximum value in criterion j will require the recalculation of normalized values. Also, when the Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 6 numerator or denominator values change, all normalized values to be generated by the techniques in Table 1 will change in criterion j. Another critical issue is the presence of the criterion that has negative values in the decision problem. The presence of negative values in a criterion can prevent effective solutions in N1 and N2 depending on the nature of the problem. The negative and positive values in N1 can be said to be mutually offsetting. Also, the negative performance values (xij) in N2 cause negative normalized values. And these situations can prevent the realization of effective solutions. The range of normalized values obtained by the normalization techniques in Table 1 are different from each other. The sum of all normalized values is equal to 1 in N1 for benefit-orientation criteria and N3 for all type criteria. Also, N1, N2, N3 are expected to generate nij in the range of 0-1. From these techniques, N3 was found to be useful if the values in a criterion are quite different from each other (Jahan & Edwards, 2015, p. 338). Maximum/minimum value-based normalization techniques are said to be less successful than the sum-based normalization techniques in handling the scale effect. Also, some of the techniques in this class cannot be effectively applied to cost criteria. In some cases, the normalized values may be higher than 1 in the techniques of this category. This situation is generally undesirable in some MCDM methods (Jahan & Edwards, 2015, p. 338). Among the techniques in this class, it is aimed to provide normalization in the range of 0-1, with the highest value being 1 in N4 and the lowest value being 1 in N5. After normalization using N6, the best performance value depending on the optimization orientation of criteria is expected to equal 1 while all normalized values are expected to fall within the range of 0-1. In N7, the normalized values are expected to be in the range of 0-1 while the best value is equal to 1. Since N8 produces large normalized values, it does not seem appropriate for most MCDM methods. The range of normalized values obtained in N9 can be expected to be smaller than the normalized values created by most techniques. N10 is one of the most used techniques in the normalization steps of MCDM methods. Providing range-based normalization, N10 is successful in handling the scale effect (Jahan & Edwards 2015, p. 338). Normalized values in N10 are expected to be in the range of 0 and 1. Z score normalization is frequently used in the application of multivariate statistical methods. Fayazbakhsh et al. (2009) used the Z score normalization as dependent of the optimization orientation. Optimization orientation dependent Z score normalization can generate negative values, but normalized values are usually around 0. This situation restricts N11's usage in MCDM methods (Jahan & Edwards, 2015). Normalization techniques were developed to be used for specific purposes or expectations. We have highlighted these purposes and expectations for the optimization orientation dependent normalization techniques. On the other hand, we will test whether the intended or expected normalized values of these normalization techniques will always be obtained in the application section. 2.3. Normalization Techniques Independent of the Optimization Orientation For normalization techniques independent of the optimization, a specific reference value/range or a constant is used instead of the optimization orientation of the Comparative analysis of normalization techniques in the context of MCM problems 7 criteria. For the techniques in this category, one or more of maximum value, minimum value, mean, standard deviation, reference (ideal/target) value/range, adjustable constant number, and data distributions are used in the normalization process (Wu, 2002; Shih et al., 2007; Jahan et al., 2011; Jahan et al., 2012; Alpar, 2013; Saranya & Manikandan, 2013; Jahan & Edwards, 2015; Gardziejczyk & Zabicki, 2017; Aytekin, 2020). Such values as maximum and minimum are also used in the normalization of optimization orientation-dependent techniques. The normalization techniques independent of the optimization orientation have no formula change based on the optimization orientation and use the same equation for all criteria types. Normalization techniques independent of the optimization orientation are given in Table 2. Table 2. Normalization techniques independent of the optimization orientation Notation Technique All criteria References N12 Z-Score (Non- Monotonic) Normalization       2 22 ij j j x R ij n e Shih et al. (2007) N13 Comprehensive Normalization              m , min min , 1 ij j ij j ij j ii x R aks maks x R x R ij n e Jahan et al. (2011); Jahan & Edwards (2015) N14 Normalization Equalizing the Average to 1   ij ij j x n Alpar (2013) N15 Normalization Equalizing Standard Deviation to 1   ij ij j x n Alpar (2013) N16 Decimal Normalization    , 0 10 ij ij x n Saranya & Manikandan (2013) N17 Reference Based Normalization         1 , min , ij j ij ij j ij jii x R n Maks maks x R Min x R Jahan et al. (2012); Jahan & Edwards (2015) N18 Wu's Reference Based Normalization    ij j ij ij j i x R n maks x R Wu (2002); Jahan & Edwards (2015) N19 Aytekin's Reference Based Normalization        1 , 0 10 ij j ij j x R n R Aytekin (2020) N20 Range Normalization Between -1 and +1           min 2 min 2 ij ijii ij ij ij ijii maks x x x n maks x x Alpar (2013) N21 Range Normalization Between 0 and +1    min min ij iji ij ij ijii x x n maks x x Alpar (2013) In the normalization techniques in Table 2, Rj shows the reference value determined for criterion j. The reference value is the base, source, or guide point that Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 8 reflects the decision maker's preferences in the decision problem. The reference value can be identified subjectively by the decision-maker, or it can be determined with the help of scientific tools or techniques (Aytekin, 2020). The arithmetic mean is generally used as a reference value in N12. The normalized values obtained in N12 are mainly in the range of 0 to ± 3. Jahan et al. (2011) used N13 in the extension of VIKOR. N14 and N15 allow the decision-maker to adjust the performance values within the criteria according to the average or standard deviation. The ρ parameter in N16 is determined by the digit value of the largest absolute number in the decision matrix. Thus, N16 can generate normalized values are between -1 and +1. N17, N18, and N19 are techniques that provide normalization based on reference. Jahan et al. (2012), in their study on material selection, proposed N17, which is the extension of Weitendorf's Linear Normalization, based on the reference value. Wu (2002) suggested using N18 in the Gray Relational Analysis if the reference value is determined among the maximum and minimum values. Aytekin (2020) proposed N19 by integrating reference-based and decimal normalization processes. In this study, N19 is revised to equate the best- normalized value to 1, and the worst normalized value to 0 according to the reference value. N20 and N21 provide normalization based on range. It is aimed to obtain normalized values between -1 and +1 in N20, and 0-1 range in N21. 2.4. Integrated-Mixed Normalization Techniques The different structures of MCDM problems always force researchers to seek new solutions. In MCDM problems, the optimization orientation-based or reference-based approaches are generally adopted for solutions. However, it may be possible for decision-makers to solve the problem using reference values for some criteria and optimization orientations for others. Similarly, some of the criteria in the decision matrix may be monotonously increasing or decreasing, while others may not. In these cases, for instance, it is possible to determine the maximum or the minimum value according to the optimization orientation as a reference value or to create a solution by applying transformations to existing techniques. In such a case, there are normalization techniques developed for these integrated/mixed situations (Zhou et al, 2006; Zeng et al., 2013; Jahan & Edwards, 2015). The integrated-mixed normalization techniques are given in Table 3. Table 3. Integrated-mixed normalization techniques Notation Technique All Criteria References N22 Ideal Linear Normalization Zhou et al (2006); Jahan & Edwards (2015)     min , for cost criteria , for benefit criteria min , , for refe , ij i ij ij ij ij i ij j ij j x x x n maks x x R maks x R  rence-based criteria            N23 Enhanced Accuracy Normalization Zeng et al. (2013); Jahan & Edwards (2015) Comparative analysis of normalization techniques in the context of MCM problems 9 Notation Technique All Criteria References              1 1 1 1 , for benefit criteria min 1 , for cost criteria min 1,96 , for criteria having normal distrubition, if 1,96 1,96 1,96                              ij ij i m ij ij i i ij ij i m ij ij i i ij ij ijm ij i i maks x x maks x x x x x x n x x x x        1 , for criteria having normal distrubition, if 1,96 1,96 1 , for criteria having normal distrubition, if 1,96                                  j ijm ij i ij x x x N22 was proposed by Zhou et al. (2006) to determine the ratio of xij to the reference value or maximum/minimum value. Zeng et al. (2013) developed N23 for an extension of VIKOR to provide practical solutions to problems in the health sector. Zeng et al. (2013) stated that the criteria having normal distribution are mostly used in the issues in the health sector and that an absolute deviation from the average is acceptable in these criteria. Also, they stated that normalization could be achieved with N23, including monotonic increasing or decreasing criteria. Apart from the normalization techniques examined in the study, there are still more normalization techniques developed for different purposes like membership functions (rough numbers, triangle, trapezoidal, etc.) which are used in the normalization processes within the framework of fuzzy logic or rough sets (Sharma et al., 2018; Vafaei et al., 2018a; Roy et al., 2019). A comparative review of the normalization techniques given in Table 1, Table 2, and Table 3 will be carried out in the following section of the study. 2.5. Comparisons of Normalization Techniques in the Literature Normalization techniques have an essential role in the solution of MCDM problems. Normalization processes used in the vast majority of MCDM methods enable the criteria of various structures to be dimensionless so that they can be directly compared. However, every normalization technique cannot be suitable for all decision problems. For example, some techniques do not provide eligible normalization for criteria with negative values or 0. Other than this, the expected range of normalized values and the rank reversal problem likely to be encountered are among the other determining factors in the selection of the normalization technique. On the other hand, it is not possible to evaluate MCDM methods independently of the normalization techniques they contain. Changing the normalization process included in an MCDM method results in the creation of a new extension/derivative of the method. There are many normalization techniques and MCDM methods in the literature. The structure of problems and the assumptions of MCDM methods are prominent factors for choosing the normalization technique. In this context, although not yet sufficient, studies on the comparison of normalization techniques have been increasing in the recent past. These studies, however, usually include a small number Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 10 of techniques and exclude most. Similarly, those that consider the selection of the normalization technique, and the criteria to be used in this selection process are limited. In one of the prominent studies in comparing normalization techniques, Çelen (2014) examined the deposit banks using the FAHP-TOPSIS integrated model. The study compared N1, N2, N4, and N10. The consistency and validity of the normalization results were evaluated under four conditions. According to the first of these conditions, the distribution of the normalized values should be similar when compared to other techniques. Under this condition, inferences could be made by looking at the mean, standard deviation, smallest-largest values, and Kolmogorov- Smirnov normality test. In the second condition, the first three and the last three of the values to be obtained by normalization techniques should be the same. The third condition states that the correlations of these rankings should be high while the fourth condition emphasizes the need for the normalization techniques to produce similar scores, and this condition was examined with correlation coefficient values. The Pearson correlation coefficient was used for measuring similarities (Çelen, 2014, p. 201-203). The use of the Kolmogorov-Smirnov normality test and Pearson correlation coefficient in the study carried out by Çelen (2014) can be seen to be unsuitable for MCDM problems. This is because MCDM problems mostly contain data structures that are not normally distributed, and the normal distribution is not generally sought in MCDM problems. Normalized values provided by normalization techniques may also have different structures. While some normalization techniques limit the data to values in a particular range, some ensure that obtained values are close to zero, while others provide only positive one-way data. It is thus clear that the structures of the decision problem and the MCDM method have a direct influence on the determination of the normalization process. The primary purpose of the normalization process performed in MCDM problems is not to obtain data with normal distribution, but to get comparable data with equal weight. However, it should be accepted that Çelen (2014) gives a different perspective on the comparison of normalization techniques. Chakraborty and Yeh (2007) suggested RCI (Ranking Consistency Index) to compare normalization techniques. In RCI, a normalization technique is evaluated based on ranking consistency with other normalization techniques. For this, it is necessary to simulate the decision matrix of at least 4x4 and at most 20x20. Finally, the results of the different normalization techniques are analyzed. In another study conducted for the selection of normalization techniques, Vafaei et al. (2018a) discussed the appropriate normalization technique for TOPSIS using the reviews previously held in the literature. Accordingly, RCI, mean and distribution measures of normalized values, Kolmogorov-Smirnov normality test, ranking consistency of normalization techniques in terms of the first three and last three rows, Pearson and Spearman correlations were used in the comparison of normalization techniques and order of suitability for TOPSIS. The authors concluded that Vector Normalization is the best technique for TOPSIS in similarity with the study of Chakraborty and Yeh (2009). They also stated that comparison based on the normal distribution is questionable (Vafaei et al., 2018a). Some of the processes mentioned in the comparison of normalization techniques do not always seem to be possible due to the structural features of MCDM problems. It is often not possible for the criteria to have a normal distribution. Furthermore, when the number of normalization techniques to be compared is high, and the size of the decision matrix exceeds 20x20, the use of RCI may not be effective. Another critical problem is that using the Pearson correlation coefficient to examine the correlation of Comparative analysis of normalization techniques in the context of MCM problems 11 the rankings does not give accurate results. Consequently, in the application part of this study, the Spearman rank correlation coefficient will be used to analyze the rank correlations of the normalization techniques. The ability to give effective normalization in different criteria structures, differences between normalized values, and their use in MCDM methods will also be examined. Normalization is one of the essential process steps in MCDM methods, and it directly affects the solution to the problem. The normalization process used in the MCDM methods should not be considered independent of the method itself. For example, TOPSIS uses Euclidean distance in solving the decision problem. In this context, vector normalization, which is the second moment according to the starting vector (0), is used in the normalization process in TOPSIS. Like TOPSIS, many other MCDM methods also have normalization procedures for the intended solution. Table 4 shows the MCDM methods and the normalization techniques used in the original forms of these methods (the first form, not extension form) and the studies comparing them depending on different normalization techniques. Table 4. Normalization in MCDM methods Methods Normalization Process in the Original Method Recommended Most Compatible Normalization Process Normalization Techniques Compared Source AHP N1 N4 + N1 N2, N4, N5, N10, N12 Vafaei et al. (2016), COPRAS N1 N1, N2 Özdağoğlu (2013) GRA N10 N2 N2, N7, N9 Chatterjee and Chakraborty (2014) TOPSIS N2 N2 N1, N4, N10 Chakraborty and Yeh (2009), Çelen (2014), Vafaei et al. (2018a) VIKOR N10 N13, N23 N13, N23 Jahan et al. (2011), Zeng et al. (2013) PROMETHEE- II N10 N2 N2, N7, N9 Chatterjee and Chakraborty (2014) MAUT - N1 N1, N2, Tchebycheff Yoon and Kim (1989) MOORA N2 N1, N2, N4, N5 N1, N4, N5, N10, N12 Özdağoğlu (2014) ELECTRE-II N2 - N1, N10 Pavličić (2001) SAW - N1, N2, N4 N1, N2, N3, N10, Fuzzy Trapezoid Membership Function Chakraborty and Yeh (2007), Vafaei et al. (2018b) As seen in Table 4, the most recommended normalization techniques are Sum- Based Linear Normalization and Vector Normalization. The normalization technique should be chosen by considering the nature of the decision problems. In the following section of the study, an applied comparison of the normalization techniques will be given. Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 12 3. Applied Comparative Analysis of Normalization Techniques with Different Scenarios In this part of the study, a comparison of normalization techniques will be highlighted. The twenty-three normalization techniques in the second section will be compared over a randomly and purposefully generated data set for a valid comparison. These sets were created by considering different scenarios to reflect the general structure of MCDM problems. The literature and scenarios were taken as a basis in determining the number of criteria and alternatives for decision matrices. To this end, studies that conducted the literature review of MCDM methods were examined. The conclusion is that the most observed numbers were four for criteria, and five for alternatives (Durucasu et al., 2017). The numbers in this study were considered important as they provide guidance. For this study, however, it was decided that it would be appropriate to have six criteria and six alternatives to reflect the structural differences of scenarios and decision problems to be used in comparing normalization techniques. After determining the number of criteria, and the number of alternatives, the different scenarios to be created were decided. Each scenario is named with a different set. To ensure that the criteria in Set 1 are different from each other, K1, whose variation range is quite wide compared to other criteria, K2 with a variation range of 0-1, K3 containing 0 and positive values, K4 containing negative values and 0, K5 containing negative, 0 and positive values together, and K6 whose values are all negative were established. While the optimization directions of all criteria in Set 1 were determined as benefit (maximum), it was evaluated as cost (minimum) in Set 2. In set 3, a scenario was created where all criteria have the benefit optimization orientation and where the ranges do not intersect was created. Accordingly, the ranges are 1-10 for K1; 11-100 for K2; 101-1,000 for K3; 1.001-10.000 for K4, 10.001-100.000 for K5 and 100.001-1.000.000 for K6. The optimization orientations of all criteria were set as benefit in Set 3 whereas it was determined as the cost in Set 4 which has the same decision matrix. In Set 5 to Set 11, the aim is to investigate the effects of adding and removing alternatives from the decision problem. Set 12 was created to examine whether units that can be converted into each other are normalized with the same values. In Set 12, the optimization orientations of the criteria are determined as benefit, while they are cost in Set 13. Set 14 was created based on the values obtained in scaling techniques commonly used in MCDM problems. In this regard, the ten-point direct rating scale for K1, Saaty’s Fundamental (Linear Priority) Scale for K2, Likert type scale for K3, DEMATEL Scoring Scale for K4, Semantic Scale for K5, and the hundred-point direct rating scale for K6 were used to determine the values of the alternatives. The ten-point direct rating scale allows alternatives to be evaluated in the range of 1-10. Alternatives are evaluated in the range of 1-9 by pairwise comparisons using Saaty’s Fundamental (Linear Priority) Scale. DEMATEL Scoring Scale is based on determining the interactions between the alternatives by pairwise comparisons in the range of 0-4 (1-5 in some research). Semantic Scale generates values in the range of 0-100 using binary comparisons of alternatives in the range of 0-6 (1-7 in some research). The hundred-point direct rating scale allows alternatives to be evaluated in the range of 1-100 (Saaty, 1977; e Costa & Vansnick, 1994; Wu, 2008). The fourteen sets in Table 5 will be used in the comparison of the normalization techniques. These sets contain decision matrices created for different scenarios. There are six criteria and six alternatives in these matrices. MS Excel was used to generate the performance values of the alternatives randomly. For this purpose, the formulas = Comparative analysis of normalization techniques in the context of MCM problems 13 RANDBETWEEN (lower_bound_value; upper_bound_value) and = RAND () were used. However, some values were determined purposely; for instance, in set 1, to examine the effects of 0, and in set 3 to check the effects of rank reversals in sets 6-11. The values that could be converted into each other were also purposely assigned in Set 12. To examine the rank reversal problem, SAW, which has one of the simplest and basic forms of MCDM methods, was used to solve the decision problems. Besides, MS Excel, SANNA, and SPSS 25.0 were used in the analysis. Table 5. The decision matrices for different scenarios Set Alternatives Criteria K1 K2 K3 K4 K5 K6 Set 1- 2 A1 750940 0,8675 51 -71 16 -3 A2 200772 0,0687 64 -50 -57 -24 A3 557819 0,9374 0 0 0 -43 A4 827702 0,9138 24 -41 19 -16 A5 26218 0,7912 75 -22 90 -61 A6 401846 0,5273 2 -55 -4 -31 Set 3-4 A1 8 29 276 3565 23351 352023 A2 4 79 491 7985 28023 354205 A3 6 68 783 4322 19956 401177 A4 7 32 335 6167 67964 376962 A5 6 21 275 3132 33352 151235 A6 2 98 920 9174 93170 928875 Set 5 It was created by removing A6 from Set 3. Set 6 It was created by removing A5 from Set 3. Set 7 It was created by removing A5 and A6 from Set 3. It was created by adding A7 to Set 3. Set 8 A7 10 100 1.000 10.000 100.000 1.000.000 It was created by adding A8 to Set 3. Set 9 A8 1 11 101 1.001 10.001 100.001 Set 10- 11 It was created by adding A7 and A8 to Set 3. Set 12- 13 A1 1 10 100 1.000 10.000 100.000 A2 2 20 200 2.000 20.000 200.000 A3 3 30 300 3.000 30.000 300.000 A4 4 40 400 4.000 40.000 400.000 A5 5 50 500 5.000 50.000 500.000 A6 6 60 600 6.000 60.000 600.000 Set 14 A1 2 0,1657 1 0,1719 100 5 A2 7 0,0881 2 0,1442 70 8 A3 8 0,0471 4 0,1821 55 60 A4 4 0,0368 2 0,1793 40 47 A5 9 0,2325 3 0,1355 25 70 A6 7 0,4298 5 0,1869 0 23 In Table 5, to show the effect of removing an alternative from the decision matrix in Set 5, A6, which ranked first in the solutions obtained with SAW in Set 3, was removed from the decision problem. In Set 6, A5, which took the last place in the solutions obtained with SAW in Set 3, was removed from the decision problem. In Set Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 14 7, both A5 and A6 alternatives in Set 3 are excluded from the decision matrix. In Set 8, A7, a new alternative with the best values in all criteria, was added to the decision matrix specified in Set 3. In Set 9, A8, a new alternative with the worst values in all criteria, was added to the decision problem in Set 3. Set 10 was created by simultaneously adding A7 and A8 to Set 3. Set 11 was created by changing the optimization orientations of Set 10. The values in Set 12 and Set 13 are assigned to represent values that can be converted into each other. In Set 14, the values of K2, K4, and K5 should be created by pairwise comparison. Values ranging between 0 and 1 were obtained for each alternative as a result of the operations performed using Saaty's Fundamental Scale in K2 and DEMATEL Scoring Scale in K4. These values were assigned following the structure of these scales; thus, multiple way performance analysis and comparison of normalization techniques can be made with Set 1-14. To examine the rank reversals problem, the ranking of the alternatives was obtained via SAW using the normalized values. In the ranking process, the criteria were considered to have equal weights. The best values in criteria depending on the optimization orientation are used as reference values for reference-based normalization techniques. The issues determined in the applications of normalization techniques in Sets 1-14 are shown in Table 6. The table contains information on the techniques regarding the sets in which the normalization process is not completed, the number of rank reversals, the maximum-minimum normalized values observed in the benefit/cost criteria, the ability to cope with the same values expressed in different units, and providing the same optimization orientation. The completion of the normalization process, which is one of the issues in Table 6, in all possible data types in a meaningful way (by cleaning all units and making them dimensionless) shows the robustness and usability of the technique. At the end of the normalization process, normalized values between 0 and 1 are preferred while different ranges of normalized values across criteria are undesirable. For example, having normalized values between -1 and +1 in one criterion, and between 0 and 10 in another is not desirable. Such a structure will change the effects/weights of the criteria on the decision problem and make the solution of the decision problem to become invalid. The success of the normalization techniques will be examined by looking at the maximum and minimum normalized values. Also, normalization techniques are expected to handle unit differences successfully. For example, a distance may be expressed in any of the units of kilometers-hectometers-decameters- meters-decimeters-centimeters. The unit of measurement used does not change the length of the distance. The normalization technique should therefore generate units that can be converted to each other and measure the same thing, with the same normalized values. This standpoint is considered when examining how the techniques cope with different units. The last issue in Table 6 is the test of whether the technique gives the same optimization orientation, which is often the benefit orientation, an indication of whether the normalization technique gives one-dimensional values in all criteria. Comparative analysis of normalization techniques in the context of MCM problems 15 Table 6. Issues determined within the scope of set 1- set 14 T e ch n iq u e The sets that normalization cannot be completed Number of rank reversals Benefit criteria Cost criteria Capability removing unit differences The success of providing the same optimization orientation Maximum normalized value Minimum normalized value Maximum normalized value Minimum normalized value N1 Set 2 & Set 14 11 1,406 -0,891 108 -16 Successful Successful N2 - 8 0,853 -0,710 1,710 0,178 Successful Successful N3 Set 1, Set 2 & Set 14 4 0,696 0 0,200 0,061 Fail Successful N4 Set 1 & Set 2 6 20,333 -0,633 1,633 -19,333 Successful Successful N5 Set 1, Set 2 & Set 14 4 4,563 -19,333 20,333 -3,563 Successful Successful N6 Set 1 & Set 2 4 20,333 -0,633 20,333 -3,563 Successful Successful N7 Set 1 & Set 2 6 1 -18,333 1 -18,333 Successful Successful N8 Set 1 & Set 2 4 2033,333 -63,333 2033,333 -356,250 Successful Successful N9 Set 1 & Set 2 6 413,444 0 8406,704 -0,254 Successful Fail N10 - 7 1 0 1 0 Successful Successful N11 - 16 2,039 -1,831 1,831 -1,917 Successful Successful N12 - 19 1,000 0,125 0,999 0,159 Successful Fail N13 - 10 1 0,368 1 0,368 Successful Successful N14 - 13 8,4375 -5,344 8,438 -5,344 Successful Fail N15 - 16 17,435 -2,983 3,711 -2,983 Successful Fail N16 - 0 0,980 -0,710 0,980 -0,710 Successful Fail N17 - 7 1 0 1 0 Successful Successful N18 Set 1, Set 3, Set 5-Set 10, Set 12 & Set 14 0 - - 1 0 Successful (only in cost criteria) Fail N19 - 0 1 0,226 1 0,064 Successful Successful N20 - 7 1 -1 1 -1 Successful Fail N21 - 7 1 0 1 0 Successful Fail N22 Set1 & Set 2 4 20,333 -0,633 20,333 -3,563 Successful Successful N23 - 7 1 0,325 1 0,499 Successful Successful The first column of Table 6 shows the sets in which normalization techniques could not complete all the operations. N2, N10, N11, N12, N13, N14, N15, N16, N17, N19, N20, N21, and N23 completed the normalization process in all criteria in all sets. N3, N4, N5, N6, N7, N8, N9, and N22 could not complete normalization processes in all or some of the criteria because of a zero value in K3; zero and negative values in K4, and negative, zero and positive values in K5. N1, N5, N6, and N22 could not create normalized values for A3 in Set 2 due to the error of dividing by zero. The normalization process with N3 could not be completed because of the zero value in K4 in Set 14. Normalization of the benefit criteria could not be ended with N18. The main reason for this situation in N18 is the selection of reference values depending on the optimization orientation. When the normalization techniques were analyzed in terms of rank reversals for Sets 1-14, only N16, N18, and N19 were found not to have rank reversal problems. However, a dramatic change in ρ value in only a specific criterion can cause rank reversals in N16. N18, which achieved normalization in only four sets, is subject to rank reversals due to maximum and minimum values in the decision matrix. In N19, if the reference and ρ values are determined depending on the first decision matrix or independently from the decision matrix, rank reversals are not observed. However, if the reference and ρ values change with changes in the decision matrix, rank reversals should be expected. Rank reversal problems were observed in other normalization techniques. Among these, N12 had the highest number of rank reversals. Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 16 Table 6 also shows the range of the values obtained by normalization techniques in benefit criteria can also be observed. Accordingly, techniques providing normalization in the range of 0-1 are N3, N10, N12, N13, N17, N19, N21, and N23. N2 and N16 give normalization between -1 and +1. The sum of normalized values from the normalization performed in N1 on the benefit criteria is expected to be 1. On the other hand, N1 generated 1.406 and -0.891 normalized values in K5 in Set 1. The following techniques were also seen to have negative normalized values; N1, N2, N4, N5, N6, N7, N8, N9, N11, N14, N15, N16, N20, and N22. In general, normalized values are expected to be within a certain range in all criteria as a result of normalization. At this point, it is more preferred that the normalized values are in the range of 0-1. In Set 1- Set 14, the average of normalized values obtained by normalization techniques was 0.7057, except for N8 and N9, which can produce structurally high normalized values. For example, an examination of the normalized values in benefit criteria reveals that N4, N5, N6, N8, N9, N14, N15, and N22 could produce normalized values larger than five by absolute value. From these techniques, N8 and N9 produced large normalized values; 2033,33 and 413,44, respectively. When the normalized values of the cost criteria were analyzed, N3, N10, N12, N13, N17, N18, N19, N21, and N23 were found to provide normalization in the range of 0-1 whereas N1, N4, N5, N6, N7, N8, N9, N11, N14, N15, N16, N20, and N22 gave negative normalized values. N1, N4, N5, N6, N7, N8, N9, N14, and N22 generated normalized values larger than five by absolute value. Among these techniques, N8 and N9 produced four-digit normalized values. Set 12 and Set 13 were created to examine whether units that could be transformed into each other were normalized in the same way. Table 6 shows that technique N3 failed in this regard. Another important comparison point is that normalized values can be used as-is in MCDM methods without additional processing. Although a significant majority of MCDM methods involve processing steps according to the cost- benefit optimization orientation, the normalization technique should shorten this process. In this context, techniques providing one-dimensionality; N1, N2, N3, N4, N5, N6, N7, N8, N10, N11, N13, N19, N22, and N23. The applications performed in Sets 1 – 14 provide an opportunity to see the positive and negative features of normalization techniques. In the following section, a general evaluation of each technique will be given. N1 does not provide useful normalization in the cost criteria as well as it does in the benefit criteria. N1 cannot complete the normalization process for cells that have 0 in the cost criteria. Besides, N1 produced quite large normalized values, such as 108 and -16, in cost criteria. However, as can be seen in K5 in Set 1, if negative, 0, and positive values are included in the benefit criteria, N1 can produce negative normalized values or values greater than 1. Also, N1 is prone to rank reversal problems. N1 can give values corresponding to the performances of the alternatives in the range of 0-1 in cases where the optimization of the criterion benefit orientation and all values are positive. Thus, the weight of an alternative in the relevant criterion can be easily determined. Besides, N1 was also found to have successfully removed unit differences. N2 produced negative values in Set 1 in the normalization of K4, K5, and K6. It also generated normalized values larger than 1 in Set 2. The main reason for this is the negative values in the decision matrix. N2 is also prone to rank reversal problems as it uses the sum of squares of the decision matrix elements. On the other hand, N2 completed the normalization processes in all criteria. N2 was also found to have Comparative analysis of normalization techniques in the context of MCM problems 17 successfully removed unit differences and gave normalized values in the benefit orientation. N3 could not complete the normalization of the criteria having zero and negative values. Although N3 provided normalization in the range of 0-1, most of the values tended to be closer to zero. The rank reversal problem in N3 was less than many other techniques. It was determined that N3 could not successfully remove unit differences. On the other hand, N3 can give normalized values in the benefit orientation. Another critical point is that N3 provides non-linear normalization. Most normalization techniques have a monotonous increasing / decreasing structure. N3 can be said to be an option for decision problems with a different structure. N4 cannot perform normalization in criteria where zero is the maximum value. Also, it produces negative normalized values in criteria having zero, negative, and positive values together. N4 gives quite large normalized values only for criteria including negative values. There may be rank reversals with N4. On the other hand, it was seen that N4 successfully removed unit differences and gave normalized values in the benefit orientation. In the cost criteria, N5 cannot complete normalization when 0 is the minimum value whereas it gives normalized value as 0 for all other values in the decision matrix. The use of the minimum value in the decision matrix in the normalization process with N5 can lead to rank reversal problems. N5 can produce quite large normalized values in criteria that have negative and zero values. However, it gives the same normalized values if the same value is expressed in different units. It also gives all normalized values in the benefit orientation. N6 could not complete normalization operations for K4 containing negative values and zero in Set 1. Similarly, in Set 2, N6 could not provide normalization for zero value in K3, K4 and K5 due to cost optimization orientations of these criteria. Also, it gave quite large normalized values such as 20,33 in K6, which had negative values in Set 1 and Set 2. Another negative feature of N6 is that it is prone to the rank reversal problem. However, N6 was found to provide normalization in the range of 0-1 in criteria where all values are positive. It is also successful in coping with different units and providing all normalized values in the benefit orientation. At first glance, N7 can be assumed not to produce negative normalized values because it operates with absolute value. On the other hand, negative normalized values for K5 and K6 were observed in Set 1 and Set 2. Normalized values such as - 18.33 produced in the mentioned sets are also quite high. N7 provided normalization in the range of 0-1 in other sets. As N7 is dependent on the maximum and minimum values in the decision matrix in the normalization processes, it is prone to the rank reversal problem. On the other hand, N7 is successful in dealing with the unit's differences and providing all normalized values in the same optimization orientation. N8 did not provide normalization for all cells in K4 in Set 1, and the cells have zero values in K4, K5, and K6 in Set 2. Also, while N8 is expected to produce 100 as normalized values for the best value according to the optimization orientations of the criteria, quite large normalized values such as -356.25 and 2033.33 values were observed in Set 1 and Set 2. The rank reversal problem was also identified in applications done in Sets 1- 4. Outstanding positive features of N8 include removing the unit's differences and giving all normalized values in the same optimization orientation. Another technique that is prone to the problem of rank reversal is N9. N9 could not complete normalization in K4, where zero is the maximum value in Set 1 and Set 2. However, the normalized values created for Set 1 and Set 2 had quite high values, such as 413.44 and 8406.70 and negative values. N9 was found to be able to remove the Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 18 unit's differences. On the other hand, it was unable to provide the benefit orientation in all criteria. In this context, the same ranks were seen for Set 12 and Set 13, in which their optimization orientations are different. Another critical point is that N9 produces non-linear normalized values. It can therefore be stated that N9 is an option for decision problems with different structures. N10 provided normalization in the range between 0 and 1 in all sets. N10 was also successful in dealing with the unit's differences and providing benefit orientation values. Among the normalization techniques, N10 is one of the most robust and successful, but it can cause rank reversals. N11 provided normalization in all sets. The maximum and minimum normalized values observed with N11 are 2.03 in Set 9 and -1.91 in Set 4. Also, the rank reversal problem was observed in normalizations with N11. On the other hand, it was successful in removing unit differences and generating normalized values in benefit criteria. N11 is one of the techniques with a non-monotonous structure. It can be stated that N11 creates an option for decision problems with different structures with these features. Techniques N1 to N11 perform the normalization process according to the optimization orientation, while N12 to N21 provide normalization independent of the optimization orientation. In normalization techniques independent of the optimization orientation, specific parameters such as reference, mean, digit value, range, standard deviation, or fixed values are used for normalization. However, this study used the maximum/minimum values as reference values following the optimization orientations of the criteria. Furthermore, when the references are determined independently from the decision matrix, the reactions of the techniques will be reported. The arithmetic mean is used as the reference value in normalization processes performed in Sets 1 - 14 with N12. In this case, normalized values ranged from 0-1 were obtained in all sets. In some sets, the same normalized values were generated for criteria with different values. At this point, it should be noted that the values in certain ranges under the normal distribution curve have the same normalized values, and a non-monotonic process is performed. Considering that most normalization techniques have a monotonous increasing/decreasing structure, N12 can be said to differ significantly from other techniques. On the other hand, these features, which distinguish N12 from other techniques, prevent it from being successful at giving all values in the benefit orientation. Furthermore, N12 was the technique in which the rank reversal problem was common in Sets 1- 14. On the other hand, N12 has the capability of removing unit differences. N13 was observed to cause the rank reversal problem. On the other hand, it provides normalization in the range of 0-1 in all sets and successful in coping with unit differences and providing all values in benefit criteria. It gave the minimum normalized value as 0.3678 in all sets. N14 performs the normalization by equating the average value to 1. If a criterion has a large range, normalized values will not be acquired within the desired range using N14. Also, normalization based on average value can lead to rank reversal problems. N14 cannot achieve normalization in all benefit criteria. It however has the capability of removing unit differences. N15 shares the same features as N14 except for the normalization by providing the standard deviation equal to 1. In N16, the digit value of the largest absolute number in the decision matrix is taken as the basis. When the maximum or the minimum value changes dramatically in a criterion, N16 can cause the rank reversal problem. However, no rank reversal was observed in comparisons made in Sets 1- 14 with N16. N16 cannot provide all values Comparative analysis of normalization techniques in the context of MCM problems 19 in the benefit orientation, but it produces normalized values between -1 and +1 in all sets. Also, N16 is successful in removing unit differences. Although N17 provides normalization based on reference, it also uses the maximum and minimum values in the criterion. This situation can cause the rank reversal problem. N17 produces normalized values in the range of 0-1, succeeds in coping with unit differences, and provides all normalized values in the same optimization orientation. N18 was the technique with the highest number of processing errors in Sets 1- 14 since the reference values were determined depending on the optimization orientation of the criteria. Revisions were made on Set 1 and Set 3 to measure the reaction of N18 and other reference-based techniques, N12, N13, N17, and N19. For this purpose, the reference values were different from the best values in the criteria. To do this, new reference values were determined to be 10% better and 10% worse than the best values in the criteria, and normalized values were examined. N18 could not complete the normalization process if the references were 0. Also, if the reference was higher than the maximum value in the criterion, N18 produced negative and large normalized values such as -8.85 and -9.37. For the other techniques, no situation other than the issues determined in the context of Set 1-14 was encountered. Also, the fact that N18 uses the maximum value in the criterion in the normalization process can lead to the rank reversal problem. N18 can remove unit differences. On the other hand, it gives the values in the benefit orientation only if the reference is less than the minimum value in the decision matrix. N19, one of the reference-based normalization techniques, produces normalized values in the range of 0-1 in all sets, but it never creates exactly normalized values as 0. N19 does not cause the rank reversal problem if the reference and ρ values are determined depending on the first decision matrix or independently from the decision matrix. In applications carried out in the context of Sets 1- 14, the rank reversal was not observed in N19. Besides, the N19 can remove unit differences and provide all values in the benefit orientation. N20 uses the average of the maximum and minimum values and the range in the normalization process. These features, on the other hand, cause the rank reversal. Unable to provide all values in the benefit orientation, N20 produces normalized values between -1 and +1. Also, N20 succeeded to cope with unit differences. N21 applies the formula used for the benefit orientation of N10 in the same way in all criteria. Being independent of the optimization orientation prevents N21 from providing all values in the benefit- orientation. N21 provided normalization in the range of 0-1 in all sets and N21 caused the rank reversal problem at the same time, but it managed to cope with unit differences. Normalization techniques from N1 to N21 provide normalization either by considering the optimization orientation of the criteria or reference or specific values. The emergence of integrated-mixed normalization techniques in the literature has given researchers different perspectives. In integrated-mixed normalization techniques, the optimization orientations of the criteria or reference/specific values are used under certain conditions. With N22, which is one of the integrated-mixed normalization techniques, normalization could not be achieved in cells that had zero value in Set 1 and Set 2. Also, quite large normalized values, such as 20.33 and -3.56, were observed in normalization depending on the optimization orientation, similar to the techniques using maximum and minimum values in criteria. N22 can cause the rank reversal problem, but it can remove unit differences and gives all normalized values in the benefit orientation. Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 20 N23 gives normalized values in the range of 0-1 in all sets and is also successful in dealing with unit differences and providing all normalized values in the benefit orientation. Additionally, it produces non-monotonic normalization, but it can cause the rank reversal problem. obtained by normalization techniques. The similarities between these rankings were examined using the Spearman rank correlation coefficient, and the results are presented in Table 7. An examination of the correlation values in Table 7 reveals that the techniques that did not complete all normalization process in criteria having the benefit optimization orientation have a low or meaningless correlation with the other techniques. The same can be said for the techniques not having a linear structure. Among the twenty-three techniques, N12 has the lowest correlation with others. N3, N9, N14, N15, N16, N20, and N21 have a low level of correlation with the other techniques. As seen in Table 7, the technique that has the most inverse correlation with other techniques is N18. This can be attributed to the low number of normalization processes completed by N18 in Sets 1-14. The other techniques were found to provide a high level of correlation (rs> 0.8) with each other in general. 4. Conclusions Normalization is a scaling process that is frequently used in creating the data structure required for the method to be used in quantitative studies. The normalization process directly affects the results of the analyzes to be performed. However, it wouldn’t be right to use traditional normalization techniques for such a process without question. In MCDM methods, normalization is used to obtain criteria that have the same weight, are dimensionless, and are suitable for compensatory processes. Although the vast majority of MCDM methods provide normalization within themselves, the nature of the decision problems makes it necessary to review these processes. On the other hand, changing the normalization process of an MCDM method partially leads to the emergence of a new extension of the relevant method. To this end, this study aimed to outline the positive and negative features of the normalization techniques that are widely used and can be used in MCDM problems, thus giving decision-makers or researchers an insight into the situations in which the considered normalization techniques can work well and where they should not be used. Eleven techniques, based on optimization orientations of criteria, ten techniques independent of optimization orientations, and two integrated-mixed techniques were evaluated in fourteen sets, reflecting different criteria and data structures considered for normalization. In the comparison, the emphasis was made on the ranges of normalized values obtained by the techniques, the presence of the rank reversal problem, normalization performance in criteria with different structures, the capacity to remove unit differences, the ability to give all normalized values in the same optimization orientation, and completion of normalization in all sets. The comparisons in Sets 1-14 showed the importance of the normalization technique selection process. Normalization may not be completed if the normalization technique is not chosen according to the data structure, and should that be the case then the validity of the results may be disputable. The results showed that some normalization techniques are not able to achieve the purposes of development or use in some cases. N1, N3, N4, N5, N6, N7, N8, N9, and N22 could not complete the normalization procedures in all or part of the criteria with zero or negative values in the decision Comparative analysis of normalization techniques in the context of MCM problems 21 matrix. N1, N4, N5, N6, N7, N8, N9, N14, N15, and N22 produced quite large normalized values. Also, they did not produce normalized values within a certain range, such as 0- 1, in all criteria. These techniques provide very high normalized values due to the criteria having a different structure in Set 1 and Set 2. The rank reversal problem was not observed in techniques N16 and N19 in Sets 1-14. Indeed, if the reference and ρ values cannot be changed after being determined at the beginning of the problem, N19 stands out as the only technique that does not cause any rank reversal problems. Also, N19 does not use values such as maximum, minimum that will change with the change in the decision matrix. Normalization techniques have different structures. However, the nature of the decision matrix, preferences of the decision-maker, and the properties of the MCDM method to be used in the solution of the decision problem should also be considered in the selection of the normalization technique. If the aim is to choose the highest performance value in a criterion and avoid the alternative with the lowest performance value, or vice versa, it would be correct to use techniques that provide normalization depending on the optimization orientation. If the decision-maker has reference/ideal/utopic values determined for each criterion, it will be necessary to use reference-based normalization techniques. If the decision-maker has the opinion that the values in the criteria do not represent the monotonous increasing/decreasing benefit/cost, then the non-monotonic or non-linear techniques should be used. However, some data, that have zero and negative values, may prevent the use of some normalization techniques, as determined in the application section. Also, the problem of rank reversal, the ranges of normalized values, the capability of removing unit differences, and the ability to give all normalized values in the same orientation, robustness and validity of the results will affect the choice of the normalization technique. This study presents a comparison of normalization techniques in different criteria structures. It sought to highlight the positive and negative features of the techniques in question thereby guiding decision-makers or researchers on the selection of techniques. The study is also envisioned to provide a perspective on the development of new normalization techniques and the creation of new extensions of MCDM methods by replacing the normalization techniques originally included in the MCDM methods with other techniques. Normalization techniques continue to be applied in most areas where data analysis is required. It is, therefore, necessary to conduct comparisons in other fields such as data mining as well. It may also be beneficial for future research to examine the normalization processes associated with fuzzy and rough sets used in MCDM problems. Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 22 Alternatives in Sets 1- 14 have been ranked by SAW using the normalized values T a b le 7 . R a n k c o rr e la ti o n s o f n o rm a li z a ti o n t e ch n iq u e s w it h e a ch o th e r in t h e c o n te x t o f sa w r a n k in g N 1 N 2 N 3 N 4 N 5 N 6 N 7 N 8 N 9 N 1 0 N 1 1 N 1 2 N 1 3 N 1 4 N 1 5 N 1 6 N 1 7 N 1 8 N 1 9 N 2 0 N 2 1 N 2 2 N 2 3 N 1 1 ,0 0 0 ,9 6 7 ** ,6 3 8 ** ,8 9 4 ** ,9 0 7 ** ,9 1 1 ** ,8 9 4 ** ,9 1 1 ** ,4 1 0 ** ,8 9 7 ** ,8 9 4 ** -0 ,0 7 2 ,9 0 2 ** ,5 2 2 ** ,4 2 9 ** ,4 0 9 ** ,8 9 7 ** -, 8 7 1 ** ,8 9 8 ** ,4 0 8 ** ,4 0 8 ** ,9 1 1 ** ,8 7 8 ** N 2 ,9 6 7 ** 1 ,0 0 0 ,6 4 4 ** ,9 2 4 ** ,8 9 2 ** ,9 0 4 ** ,9 2 4 ** ,9 0 4 ** ,4 0 2 ** ,9 0 0 ** ,9 0 3 ** -0 ,0 4 6 ,8 9 1 ** ,3 9 5 ** ,3 4 5 ** ,3 4 3 ** ,9 0 0 ** -, 8 4 2 ** ,9 1 4 ** ,3 4 2 ** ,3 4 2 ** ,9 0 4 ** ,8 7 7 ** N 3 ,6 3 8 ** ,6 4 4 ** 1 ,0 0 0 ,6 4 3 ** ,5 5 4 ** ,6 2 9 ** ,6 4 3 ** ,6 2 9 ** ,2 5 0 * ,6 2 1 ** ,6 4 7 ** 0 ,1 8 4 ,6 3 5 ** ,2 4 8 * ,2 4 5 * 0 ,2 0 6 ,6 2 1 ** -, 5 5 0 * ,5 8 3 ** ,2 4 4 * ,2 4 4 * ,6 2 9 ** ,6 2 5 ** N 4 ,8 9 4 ** ,9 2 4 ** ,6 4 3 ** 1 ,0 0 0 ,9 0 7 ** ,9 8 3 ** 1 ,0 0 0 ** ,9 8 3 ** ,4 3 5 ** ,9 4 6 ** ,9 5 2 ** -0 ,0 0 8 ,9 5 9 ** ,3 7 8 ** ,4 2 0 ** ,4 3 3 ** ,9 4 6 ** -, 8 8 5 ** ,9 5 7 ** ,4 1 3 ** ,4 1 3 ** ,9 8 3 ** ,9 2 9 ** N 5 ,9 0 7 ** ,8 9 2 ** ,5 5 4 ** ,9 0 7 ** 1 ,0 0 0 ,9 2 5 ** ,9 0 7 ** ,9 2 5 ** ,3 6 0 ** ,9 4 0 ** ,9 1 2 ** 0 ,0 3 1 ,9 2 1 ** ,3 5 5 ** ,3 7 4 ** ,3 8 9 ** ,9 4 0 ** -, 8 7 1 ** ,9 5 4 ** ,3 7 5 ** ,3 7 5 ** ,9 2 5 ** ,9 1 7 ** N 6 ,9 1 1 ** ,9 0 4 ** ,6 2 9 ** ,9 8 3 ** ,9 2 5 ** 1 ,0 0 0 ,9 8 3 ** 1 ,0 0 0 ** ,4 5 2 ** ,9 4 2 ** ,9 3 6 ** 0 ,0 0 0 ,9 5 5 ** ,3 9 9 ** ,4 3 7 ** ,4 3 7 ** ,9 4 2 ** -, 8 7 1 ** ,9 5 3 ** ,4 1 7 ** ,4 1 7 ** 1 ,0 0 0 ** ,9 2 5 ** N 7 ,8 9 4 ** ,9 2 4 ** ,6 4 3 ** 1 ,0 0 0 ** ,9 0 7 ** ,9 8 3 ** 1 ,0 0 0 ,9 8 3 ** ,4 3 5 ** ,9 4 6 ** ,9 5 2 ** -0 ,0 0 8 ,9 5 9 ** ,3 7 8 ** ,4 2 0 ** ,4 3 3 ** ,9 4 6 ** -, 8 8 5 ** ,9 5 7 ** ,4 1 3 ** ,4 1 3 ** ,9 8 3 ** ,9 2 9 ** N 8 ,9 1 1 ** ,9 0 4 ** ,6 2 9 ** ,9 8 3 ** ,9 2 5 ** 1 ,0 0 0 ** ,9 8 3 ** 1 ,0 0 0 ,4 5 2 ** ,9 4 2 ** ,9 3 6 ** 0 ,0 0 0 ,9 5 5 ** ,3 9 9 ** ,4 3 7 ** ,4 3 7 ** ,9 4 2 ** -, 8 7 1 ** ,9 5 3 ** ,4 1 7 ** ,4 1 7 ** 1 ,0 0 0 ** ,9 2 5 ** N 9 ,4 1 0 ** ,4 0 2 ** ,2 5 0 * ,4 3 5 ** ,3 6 0 ** ,4 5 2 ** ,4 3 5 ** ,4 5 2 ** 1 ,0 0 0 ,4 2 6 ** ,4 2 3 ** -0 ,0 5 2 ,4 3 9 ** ,9 2 6 ** ,9 5 5 ** ,9 4 2 ** ,4 2 6 ** ,9 8 6 ** ,4 1 4 ** ,9 5 9 ** ,9 5 9 ** ,4 5 2 ** ,4 1 0 ** N 1 0 ,8 9 7 ** ,9 0 0 ** ,6 2 1 ** ,9 4 6 ** ,9 4 0 ** ,9 4 2 ** ,9 4 6 ** ,9 4 2 ** ,4 2 6 ** 1 ,0 0 0 ,9 7 5 ** -0 ,0 0 3 ,9 8 2 ** ,3 4 1 ** ,3 9 9 ** ,3 5 3 ** 1 ,0 0 0 ** -, 9 0 9 ** ,9 4 9 ** ,4 0 2 ** ,4 0 2 ** ,9 4 2 ** ,9 4 9 ** N 1 1 ,8 9 4 ** ,9 0 3 ** ,6 4 7 ** ,9 5 2 ** ,9 1 2 ** ,9 3 6 ** ,9 5 2 ** ,9 3 6 ** ,4 2 3 ** ,9 7 5 ** 1 ,0 0 0 0 ,0 0 7 ,9 6 8 ** ,3 3 8 ** ,4 0 2 ** ,3 5 6 ** ,9 7 5 ** -, 8 7 5 ** ,9 2 9 ** ,3 9 9 ** ,3 9 9 ** ,9 3 6 ** ,9 2 4 ** N 1 2 - 0 ,0 7 2 -0 ,0 4 6 0 ,1 8 4 -0 ,0 0 8 0 ,0 3 1 0 ,0 0 0 -0 ,0 0 8 0 ,0 0 0 -0 ,0 5 2 -0 ,0 0 3 0 ,0 0 7 1 ,0 0 0 -0 ,0 3 1 -0 ,0 8 9 0 ,0 2 7 -0 ,0 5 1 -0 ,0 0 3 0 ,0 0 1 -0 ,0 1 7 -0 ,0 0 6 -0 ,0 0 6 0 ,0 0 0 -0 ,0 1 6 N 1 3 ,9 0 2 ** ,8 9 1 ** ,6 3 5 ** ,9 5 9 ** ,9 2 1 ** ,9 5 5 ** ,9 5 9 ** ,9 5 5 ** ,4 3 9 ** ,9 8 2 ** ,9 6 8 ** -0 ,0 3 1 1 ,0 0 0 ,3 5 7 ** ,4 0 0 ** ,3 5 8 ** ,9 8 2 ** -, 8 9 8 ** ,9 3 5 ** ,3 9 2 ** ,3 9 2 ** ,9 5 5 ** ,9 4 7 ** N 1 4 ,5 2 2 ** ,3 9 5 ** ,2 4 8 * ,3 7 8 ** ,3 5 5 ** ,3 9 9 ** ,3 7 8 ** ,3 9 9 ** ,9 2 6 ** ,3 4 1 ** ,3 3 8 ** -0 ,0 8 9 ,3 5 7 ** 1 ,0 0 0 ,9 0 0 ** ,8 9 3 ** ,3 4 1 ** ,9 1 1 ** ,3 3 9 ** ,8 8 1 ** ,8 8 1 ** ,3 9 9 ** ,3 1 3 ** N 1 5 ,4 2 9 ** ,3 4 5 ** ,2 4 5 * ,4 2 0 ** ,3 7 4 ** ,4 3 7 ** ,4 2 0 ** ,4 3 7 ** ,9 5 5 ** ,3 9 9 ** ,4 0 2 ** 0 ,0 2 7 ,4 0 0 ** ,9 0 0 ** 1 ,0 0 0 ,9 0 9 ** ,3 9 9 ** ,9 6 6 ** ,3 6 0 ** ,9 7 5 ** ,9 7 5 ** ,4 3 7 ** ,3 6 8 ** N 1 6 ,4 0 9 ** ,3 4 3 ** 0 ,2 0 6 ,4 3 3 ** ,3 8 9 ** ,4 3 7 ** ,4 3 3 ** ,4 3 7 ** ,9 4 2 ** ,3 5 3 ** ,3 5 6 ** -0 ,0 5 1 ,3 5 8 ** ,8 9 3 ** ,9 0 9 ** 1 ,0 0 0 ,3 5 3 ** ,9 6 6 ** ,4 0 8 ** ,9 2 9 ** ,9 2 9 ** ,4 3 7 ** ,3 3 4 ** N 1 7 ,8 9 7 ** ,9 0 0 ** ,6 2 1 ** ,9 4 6 ** ,9 4 0 ** ,9 4 2 ** ,9 4 6 ** ,9 4 2 ** ,4 2 6 ** 1 ,0 0 0 ** ,9 7 5 ** -0 ,0 0 3 ,9 8 2 ** ,3 4 1 ** ,3 9 9 ** ,3 5 3 ** 1 ,0 0 0 -, 9 0 9 ** ,9 4 9 ** ,4 0 2 ** ,4 0 2 ** ,9 4 2 ** ,9 4 9 ** N 1 8 -, 8 7 ** -, 8 4 2 ** -, 5 5 0 * -, 8 8 5 ** -, 8 7 1 ** -, 8 7 1 ** -, 8 8 5 ** -, 8 7 1 ** ,9 8 6 ** -, 9 0 9 ** -, 8 7 5 ** 0 ,0 0 1 -, 8 9 8 ** ,9 1 1 ** ,9 6 6 ** ,9 6 6 ** -, 9 0 9 ** 1 ,0 0 0 -, 8 9 8 ** 1 ,0 0 0 ** 1 ,0 0 0 ** -, 8 7 1 ** -, 8 7 5 ** N 1 9 ,8 9 8 ** ,9 1 4 ** ,5 8 3 ** ,9 5 7 ** ,9 5 4 ** ,9 5 3 ** ,9 5 7 ** ,9 5 3 ** ,4 1 4 ** ,9 4 9 ** ,9 2 9 ** -0 ,0 1 7 ,9 3 5 ** ,3 3 9 ** ,3 6 0 ** ,4 0 8 ** ,9 4 9 ** -, 8 9 8 ** 1 ,0 0 0 ,3 5 7 ** ,3 5 7 ** ,9 5 3 ** ,9 2 6 ** N 2 0 ,4 0 8 ** ,3 4 2 ** ,2 4 4 * ,4 1 3 ** ,3 7 5 ** ,4 1 7 ** ,4 1 3 ** ,4 1 7 ** ,9 5 9 ** ,4 0 2 ** ,3 9 9 ** -0 ,0 0 6 ,3 9 2 ** ,8 8 1 ** ,9 7 5 ** ,9 2 9 ** ,4 0 2 ** 1 ,0 0 0 ** ,3 5 7 ** 1 ,0 0 0 1 ,0 0 0 ** ,4 1 7 ** ,3 7 1 ** N 2 1 ,4 0 8 ** ,3 4 2 ** ,2 4 4 * ,4 1 3 ** ,3 7 5 ** ,4 1 7 ** ,4 1 3 ** ,4 1 7 ** ,9 5 9 ** ,4 0 2 ** ,3 9 9 ** -0 ,0 0 6 ,3 9 2 ** ,8 8 1 ** ,9 7 5 ** ,9 2 9 ** ,4 0 2 ** 1 ,0 0 0 ** ,3 5 7 ** 1 ,0 0 0 ** 1 ,0 0 0 ,4 1 7 ** ,3 7 1 ** N 2 2 ,9 1 1 ** ,9 0 4 ** ,6 2 9 ** ,9 8 3 ** ,9 2 5 ** 1 ,0 0 0 ** ,9 8 3 ** 1 ,0 0 0 ** ,4 5 2 ** ,9 4 2 ** ,9 3 6 ** 0 ,0 0 0 ,9 5 5 ** ,3 9 9 ** ,4 3 7 ** ,4 3 7 ** ,9 4 2 ** -, 8 7 1 ** ,9 5 3 ** ,4 1 7 ** ,4 1 7 ** 1 ,0 0 0 ,9 2 5 ** N 2 3 ,8 7 8 ** ,8 7 7 ** ,6 2 5 ** ,9 2 9 ** ,9 1 7 ** ,9 2 5 ** ,9 2 9 ** ,9 2 5 ** ,4 1 0 ** ,9 4 9 ** ,9 2 4 ** -0 ,0 1 6 ,9 4 7 ** ,3 1 3 ** ,3 6 8 ** ,3 3 4 ** ,9 4 9 ** -, 8 7 5 ** ,9 2 6 ** ,3 7 1 ** ,3 7 1 ** ,9 2 5 ** 1 ,0 0 0 *. C o rr e la ti o n i s si g n if ic a n t a t th e 0 .0 5 l e v e l (2 -t a il e d ). ** . C o rr e la ti o n i s si g n if ic a n t a t th e 0 .0 1 l e v e l (2 -t a il e d ). Comparative analysis of normalization techniques in the context of MCM problems 23 Funding: This research received no external funding. Conflicts of Interest: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. References Alpar, R. (2013). Uygulamalı çok değişkenli istatistik yöntemler. Detay Yayıncılık, Ankara. Aytekin, A. (2020). Çok kriterli karar problemine uzaklık ve referans temelli çözüm yaklaşımı [Unpublished PhD thesis]. Anadolu University, Eskişehir. Brauers, W. K., & Zavadskas, E. K. (2006). The MOORA method and its application to privatization in a transition economy. Control and Cybernetics, 35, 445-469. Chakraborty, S., & Yeh, C. H. (2007). A simulation based comparative study of normalization procedures in multiattribute decision making. In Proceedings of the 6th Conference on 6th WSEAS Int. Conf. on Artificial Intelligence, Knowledge Engineering and Data Bases, 6, 102-109. Chakraborty, S., & Yeh, C. H. (2009). A simulation comparison of normalization procedures for TOPSIS. In 2009 International Conference on Computers & Industrial Engineering (pp. 1815-1820). IEEE. Chatterjee, P., & Chakraborty, S. (2014). Investigating the effect of normalization norms in flexible manufacturing sytem selection using multi-criteria decision-making methods. Journal of Engineering Science & Technology Review, 7(3), 141-150. Çelen, A. (2014). Comparative analysis of normalization procedures in TOPSIS method: With an application to Turkish deposit banking market. Informatica, 25(2), 185-208. Durucasu, H., Aytekin, A., Saraç, B., & Orakçı, E (2017). Current application fields of ELECTRE and PROMETHEE: A literature review. Alphanumeric Journal, 5(2), 229-270. doi.org/10.17093/alphanumeric.320235 e Costa, C. A. B., & Vansnick J. C. (1994). MACBETH—An interactive path towards the construction of cardinal value functions. International Transactions in Operational Research, 1(4), 489-500. Gardziejczyk, W., & Zabicki, P. (2017). Normalization and variant assessment methods in selection of road alignment variants–case study. Journal of Civil Engineering and Management, 23(4), 510-523. doi.org/10.3846/13923730.2016.1210223 Fayazbakhsh, K., Abedian, A., Manshadi, B. D., & Khabbaz R.S. (2009). Introducing a novel method for materials selection in mechanical design using Z-transformation in statistics for normalization of material properties. Materials & Design, 30(10), 4396- 404. doi.org/10.1016/j.matdes.2009.04.004 Aytekin/Decis. Mak. Appl. Manag. Eng. 4 (2) (2021) 1-25 24 Jahan, A., Mustapha, F., Ismail, M. Y., Sapuan, S. M., & Bahraminasab, M. (2011). A comprehensive VIKOR method for material selection. Materials & Design, 32(3), 1215- 1221. doi.org/10.1016/j.matdes.2010.10.015 Jahan, A., Bahraminasab, M., & Edwards, K. L. (2012). A target-based normalization technique for materials selection. Materials & Design, 35: 647-654. doi.org/10.1016/j.matdes.2011.09.005 Jahan, A., & Edwards, K. L. (2015). A state-of-the-art survey on the influence of normalization techniques in ranking: Improving the materials selection process in engineering design. Materials & Design, 65, 335-342. doi.org/10.1016/j.matdes.2014.09.022 Jensen, R. E. (1984). An alternative scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 28(3), 317-332. Kainulainen, T., Leskinen, P., Korhonen, P., Haara, A., & Hujala, T. (2009). A statistical approach to assessing interval scale preferences in discrete choice problems. Journal of the Operational Research Society, 60(2), 252-258. Lootsma, F.A. (1999). Multi-criteria decision analysis via ratio and difference judgement (Vol. 29), Kluwer Academic Publishers, Dordrecht. Milani, A.S., Shanian, A., Madoliat, R., & Nemes, J.A. (2005). The effect of normalization norms in multiple attribute decision making models: a case study in gear material selection. Structural and Multidisciplinary Optimization, 29(4), 312-318. Özdağoğlu, A. (2013). Çok ölçütlü karar verme modellerinde normalizasyon tekniklerinin sonuçlara etkisi: COPRAS örneği. Eskişehir Osmangazi Üniversitesi İktisadi ve İdari Bilimler Dergisi, 8(2), 229-255. Özdağoğlu, A. (2014). Normalizasyon yöntemlerinin çok ölçütlü karar verme sürecine etkisi–MOORA yöntemi incelemesi. Ege Academic Review, 14(2), 283-294. Pavličić, D. (2001). Normalization affects the results of MADM methods. Yugoslav Journal Of Operations Research, 11(2), 251-265. Podviezko, A., & Podvezko V (2015). Influence of data transformation on multi-criteria evaluation result. Procedia Engineering, 122, 151-157. doi.org/10.1016/j.proeng.2015.10.019 Roberts, F. S. (1984). Measurement theory: With applications to decisionmaking, utility, and the social sciences (Encyclopedia of mathematics and its applications). Cambridge University Press. doi.org/10.1017/CBO9780511759871 Roy, J., Sharma, H. K., Kar, S., Zavadskas, E. K., & Saparauskas, J. (2019). An extended COPRAS model for multi-criteria decision-making problems and its application in web-based hotel evaluation and selection. Economic research-Ekonomska istraživanja, 32(1), 219-253. doi.org/10.1080/1331677X.2018.1543054 Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234-281. Saranya, C., & Manikandan, G. (2013). A study on normalization techniques for privacy preserving data mining. International Journal of Engineering and Technology (IJET), 5(3), 2701-2704. Comparative analysis of normalization techniques in the context of MCM problems 25 Sarraf, A.Z., Mohaghar, A., & Bazargani H. (2013). Developing TOPSIS method using statistical normalization for selecting knowledge management strategies. Journal of Industrial Engineering and Management, 6(4), 860-875. doi.org/10.3926/jiem.573 Sharma, H. K., Roy, J., Kar, S., & Prentkovskis, O. (2018). Multi criteria evaluation framework for prioritizing indian railway stations using modified rough AHP-MABAC method. Transport and telecommunication journal, 19(2), 113-127. doi.org/10.2478/ttj-2018-0010 Shih, H. S., Shyur H. J., & Lee E. S. (2007). An extension of TOPSIS for group decision making. Mathematical and Computer Modelling, 45(8), 801-13. doi.org/10.1016/j.mcm.2006.03.023 Tavşancıl, E. (2006). Tutumların ölçülmesi ve SPSS ile veri analizi. Ankara, Nobel. Vafaei, N., Ribeiro, R. A., & Camarinha-Matos, L. M. (2016). Normalization techniques for multi-criteria decision making: analytical hierarchy process case study. In Doctoral Conference on Computing, Electrical and Industrial Systems. Springer, Cham, 261-269. Vafaei, N., Ribeiro, R.A., & Camarinha-Matos, L.M. (2018a). Data normalisation techniques in decision making: case study with TOPSIS method. International Journal of Information and Decision Sciences, 10(1), 19-38. doi.org/10.1504/IJIDS.2018.090667 Vafaei, N., Ribeiro, R. A., & Camarinha-Matos, L. M. (2018b). Selection of normalization technique for weighted average multi-criteria decision making. In Doctoral Conference on Computing, Electrical and Industrial Systems. Springer, Cham, 43-52. Wu, H. H. (2002). A comparative study of using grey relational analysis in multiple attribute decision making problems. Quality Engineering, 15(2), 209-17. Wu, W. (2008). Choosing knowledge management strategies by using a combined ANP and DEMATEL approach. Expert Systems with Applications, 35(3), 828-835. Yoon, K., & Kim, G. (1989). Multiple attribute decision analysis with imprecise information. IIE Transactions. 21(1), 21-26. Zavadskas, E.K, & Turskis Z. (2008). A new logarithmic normalization method in games theory. Informatica. 19(2), 303-14. Zeng Q. L., Li D. D., & Yang Y. B. (2013). VIKOR method with enhanced accuracy for multiple criteria decision making in healthcare management. Journal of Medical Systems. 37(2), 9908. Zhou, P., Ang B. W., & Poh K. L. (2006). Comparing aggregating methods for constructing the composite environmental index: An objective measure. Ecological Economics, 59(3), 305-311. © 2018 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).