Identification of Outliers and Robust Estimations in the AHP IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 577 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 PRIORITY VECTOR ESTIMATION: CONSISTENCY, COMPATIBILITY, PRECISION Stan Lipovetsky 1 stan.lipovetsky@gmail.com ABSTRACT Various methods of priority vector estimation are known in the Analytic Hierarchy Process (AHP). They include the classical eigenproblem method given by Thomas Saaty, developments in least squares and the multiplicative approach, robust estimation based on transformation of the pairwise ratios to the shares of preferences, and other approaches. In this paper, the priority vectors are completed with validation of data consistency, comparisons of vectors’ compatibility, and estimation of precision for matrix approximation by vectors. The numerical results for different data sizes and consistency show that the considered methods reveal useful features, are simple and convenient, and capable of facilitating practical applications of the AHP in solving various multiple- criteria decision making problems. Keywords: AHP priority vector estimations; consistency measures; S-index and G-index of compatibility; precision of fitting 1. Introduction The Analytic Hierarchy Process (AHP) is a widely used methodology and a set of methods for solving various problems of prioritization. Developed by Thomas Saaty and expanded in numerous works by many authors, it is currently one of the main approaches for managers and practitioners who need to apply multi-criteria decision making to reach their goals. In this work, the term AHP is used not in its whole rich entirety but in a narrower sense as a method of finding local priority vectors by a pairwise comparison matrix. Estimations of priority vectors in the AHP include the classical eigenproblem method (EM) proposed by Saaty (1977, 1980, 1994, 1996, 2005), the least squares (LS) solution and the multiplicative or logarithmic (LN) least squares described in Saaty and Vargas (1984, 1994) and Lootsma (1993, 1999), and numerous other modifications (for instance, Lipovetsky, 1996, 2009, 2013; Lipovetsky and Tishler, 1999). Particularly, priority vector robust estimation (RE) which is not prone to possible inconsistencies in pairwise judgements can be based on the ratio transformation to the shares of preferences and obtained by Markov chain modeling for steady-state probabilities (Lipovetsky and Conklin, 2002, 2015). The current work presents the results of comparisons between the EM, LN, LS, and RE using several characteristics of closeness for the obtained solutions, including pair 1 Acknowledgement: I am grateful to three reviewers whOSE comments improved the work. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 578 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 correlations, the so-called Saaty compatibility index (S-compatibility) (Saaty, 2005; Saaty and Peniwati, 2007), and the Garuti compatibility index or G-compatibility (Garuti, 2007; Garuti and Salomon, 2011). For different sizes and consistency of the matrices of judgement used in the classical AHP literature, the priority vectors are calculated, their compatibility indices estimated, and characteristics of the matrix fit by the vectors are described. In general, the explored methods are simple and convenient and can significantly facilitate practical applications of the AHP for optimum solutions in various problems. The paper is organized as follows: Section 2 describes the methods of priority estimation, Section 3 defines the measures of compatibility and quality of fit, Section 4 discusses several numerical examples, and Section 5 concludes on the obtained results. 2. Priority vector estimations Let us briefly describe several main methods of priority vector estimations. The general form of the AHP pairwise priority ratios matrix can be written as follows: 𝐴 = ( 1 π‘Ž12 . . . π‘Ž1𝑛 π‘Ž21 1 . . . π‘Ž2𝑛 . . . . . . . . . . . . . . . . . . . . . . . . . . . π‘Žπ‘›1 π‘Žπ‘›2 . . . 1 ). (1) This is a Saaty matrix of pairwise judgements among n items, elicited from an expert. Each element aij shows a quotient of preference of the i-th item over the j-th item in their comparison, so we have the transposed-reciprocal elements π‘Žπ‘–π‘— = π‘Žπ‘—π‘– βˆ’1. A theoretical Saaty matrix of pair comparisons defines each ij-th element as a ratio of the unknown priorities wi and wj: π‘Š = ( 𝑀1/𝑀1 𝑀1/𝑀2 ... 𝑀1/𝑀𝑛 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 𝑀𝑛 /𝑀1 𝑀𝑛 /𝑀2 ... 𝑀𝑛 /𝑀𝑛 ) = 𝑀 βˆ— ( 1 𝑀 )β€². (2) The vector-column w consists of the elements w1, w2,…, wn, the vector-row (1/w)’ contains the reciprocal values 1/w1, 1/w2,…, 1/wn, and the right-hand side of the relationship (Equation 2) shows the outer product of these two vectors (where the prime denotes transposition). From Equation (2), it is easy to find the identical relation π‘Šπ‘€ = 𝑛𝑀 for the theoretical matrix and vector. For the obtained matrix (Equation 1), a similar relationship can be presented as the eigenproblem: 𝐴𝛼 = πœ†π›Ό , (3) where the first eigenvector alpha for the maximum eigenvalue πœ† defines the vector of priorities. This is the eigenvector method EM of the classical AHP. Another known way is the least squares estimation for the priority vector which can be expressed via the following eigenproblem: IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 579 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 (𝐴𝐴′)𝛼 = πœ†2𝛼 , (4) The main vector alpha yields the priority vector in the LS approach. The third popular approach to priority estimation is called the multiplicative or logarithmic technique. It can be reduced to calculating the elements of the priority vector as the geometric means of the elements in each row of the matrix (Equation 1): 𝛼𝑖 = √∏ π‘Žπ‘–π‘— 𝑛 𝑗=1 𝑛 . (5) The obtained AHP priority vectors are also standardized by the total of the elements, so a solution is divided by the total of all of the elements and the sum of the normalized components equals one: 𝛼𝑖 π‘›π‘œπ‘Ÿπ‘šπ‘Žπ‘™π‘–π‘§π‘’π‘‘ = 𝛼𝑖 /π‘ π‘’π‘š(𝛼𝑖 ) . (6) This is the priority vector estimation in the LN approach. The solution with robust estimation (RE) is less prone to possible inconsistencies in the pairwise judgements; let us introduce a theoretical matrix of shares as follows: π‘ˆ = ( 𝑀1/(𝑀1 + 𝑀1) 𝑀1/(𝑀1 + 𝑀2) ... 𝑀1/(𝑀1 + 𝑀𝑛 ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 𝑀𝑛 /(𝑀𝑛 + 𝑀1) 𝑀𝑛 /(𝑀𝑛 + 𝑀2) ... 𝑀𝑛 /(𝑀𝑛 + 𝑀𝑛 ) ), (7) Each element uij in Equation (7) is defined as i-th priority in the sum of i-th and j-th priorities: 𝑒𝑖𝑗 = 𝑀𝑖 𝑀𝑖+𝑀𝑗 = 𝑀𝑖/𝑀𝑗 1+𝑀𝑖/𝑀𝑗 . (8) To estimate the priority vector using the matrix (Equation 7), we write identical equalities: { 𝑀1 𝑀1+𝑀1 (𝑀1 + 𝑀1) + 𝑀1 𝑀1+𝑀2 (𝑀1 + 𝑀2)+. . . + 𝑀1 𝑀1+𝑀𝑛 (𝑀1 + 𝑀𝑛 ) = 𝑛𝑀1 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 𝑀𝑛 𝑀𝑛+𝑀1 (𝑀𝑛 + 𝑀1) + 𝑀𝑛 𝑀𝑛+𝑀2 (𝑀𝑛 + 𝑀2)+. . . + 𝑀𝑛 𝑀𝑛+𝑀𝑛 (𝑀𝑛 + 𝑀𝑛 ) = 𝑛𝑀𝑛 . (9) Then, with notation (Equation 8) we present the system (Equation 9) as: { (𝑒11 + βˆ‘ 𝑒1𝑗 𝑛 𝑗=1 )𝑀1 + 𝑒12𝑀2+. . . +𝑒1𝑛𝑀𝑛 = 𝑛𝑀1 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 𝑒𝑛1𝑀1 + 𝑒𝑛2𝑀2+. . . +(𝑒𝑛𝑛 + βˆ‘ 𝑒𝑛𝑗 𝑛 𝑗=1 )𝑀𝑛 = 𝑛𝑀𝑛   . (10) IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 580 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 In the matrix form the system (Equation 10) can be written as: (π‘ˆ + π‘‘π‘–π‘Žπ‘”(π‘ˆπ‘’))𝑀 = 𝑛𝑀, (11) where U is the matrix (Equation 7), e denotes a uniform vector of n-th order, and diag(Ue) is a diagonal matrix of totals in each row of matrix U. In the classical AHP, the pair ratios wi/wj (Equation 2) are estimated by the elicited values aij (Equation 1). Using aij in Equation (8), we obtain the empirical estimates bij of the pairs’ shares: 𝑏𝑖𝑗 = π‘Žπ‘–π‘— 1+π‘Žπ‘–π‘— . (12) This transformation of the elements of a matrix A (Equation 1) yields a pairwise share matrix B with the elements (Equation 12). These elements (Equation 12) are positive, less than one, and have a property 𝑏𝑖𝑗 + 𝑏𝑗𝑖 = 1. This means that the transposed elements bij and bji are skew-symmetrical off the diagonal bii=0.5, so 𝑏𝑖𝑗 βˆ’ 𝑏𝑖𝑖 = βˆ’(𝑏𝑗𝑖 βˆ’ 𝑏𝑖𝑖 ). For the empirical Saaty matrix A (Equation 1), we have the eigenproblem (Equation 3) in place of the theoretical relationship (Equation 2). Similarly, using the empirical skew- symmetric matrix B (Equation 12) in place of theoretical matrix U, we represent the system (Equation 11) as the eigenproblem: (𝐡 + π‘‘π‘–π‘Žπ‘”(𝐡𝑒))𝛼 = πœ†π›Ό, (13) where Ξ± as the main eigenvector. This is the RE vector of priority, and its properties have been studied in the works of Lipovetsky and Conklin (2002, 2015). 3. Measures of consistency, compatibility, and precision Due to the general methodology of the AHP, the so-called consistency index (CI) equals 𝐢𝐼 = πœ†βˆ’π‘› π‘›βˆ’1 (14) where πœ† is the maximum eigenvalue of the matrix in the problem (Equation 3), and n is the matrix order. The so-called random consistency index (RI) is a constant tabulated in the AHP for various n, and the consistency ratio (CR) equals the following value: 𝐢𝑅 = 𝐢𝐼 𝑅𝐼 . (15) A value of CR up to 10% is considered to indicate a small inconsistency in the matrix of the pairwise comparisons (Equation 1), and therefore an acceptable matrix; however, if the CR > 10%, a review of the elicited judgements could be required. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 581 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 For comparisons between the obtained solutions, several characteristics can be applied. Among those are the pairwise correlation between the elements of two vectors, which can be reduced to the expression: π‘Ÿ(π‘₯, 𝑦) = βˆ‘ (π‘₯π‘–βˆ’οΏ½Μ…οΏ½)(π‘¦π‘–βˆ’οΏ½Μ…οΏ½) 𝑛 𝑖=1 βˆšβˆ‘ (π‘₯π‘–βˆ’οΏ½Μ…οΏ½) 2𝑛 𝑖=1 βˆ‘ (π‘¦π‘–βˆ’οΏ½Μ…οΏ½) 2𝑛 𝑖=1 = βˆ‘ π‘₯π‘–π‘¦π‘–βˆ’ 1 𝑛 𝑛 𝑖=1 βˆšβˆ‘ π‘₯𝑖 2βˆ’ 1 𝑛 𝑛 𝑖=1 βˆšβˆ‘ 𝑦𝑖 2βˆ’ 1 𝑛 𝑛 𝑖=1 , (16) where a bar above the variables denotes the mean values and those equal 1/n for the vectors normalized by Equation (6). Without the items 1/n for centering (and this value is small for a bigger n), the measure (Equation 16) coincides with the cosine as a normalized projection of one vector onto another one. The closer a correlation or cosine is to 1, the higher the similarity of the two solutions. The cosine values repeat the correlations but are slightly bigger, so for a more conservative measure the correlation is preferred. Another good measure of closeness between two vectors is the so-called Saaty compatibility index (S-compatibility) (Saaty, 2005; Saaty and Peniwati, 2007; Garuti and Salomon, 2011). This index can be built as follows. For two vectors x and y of an n-th order, build a matrix X with its elements defined as quotients Xij=xi/xj of the components of the vector x, and a matrix Y with its elements defined as quotients Yij=yi/yj of the components of the vector y. Take the transposed matrix Y’ with the elements Y’ij=yj/yi and find the Hadamard element-wise product of these two matrices X*Y’, then the S- index is defined as the normalized total of the elements of this matrix: 𝑆 = 1 𝑛2 βˆ‘ 𝑋𝑖𝑗 π‘Œπ‘–π‘— ′𝑛 𝑖,𝑗=1 = 1 𝑛2 βˆ‘ π‘₯𝑖 π‘₯𝑗 𝑛 𝑖,𝑗=1 𝑦𝑗 𝑦𝑖 . (17) If two vectors coincide, this index equals 1. Within 10% of discrepancy, when 𝑆 ≀ 1.1, the vectors are considered compatible; otherwise, when 𝑆 > 1.1, they are incompatible (Saaty and Peniwati, 2007). A further development of a compatibility measure in the so-called compatibility index G was proposed in Garuti (2007) and Garuti and Salomon (2011) where it was defined as: 𝐺 = βˆ‘ π‘šπ‘–π‘›(π‘₯𝑖,𝑦𝑖) π‘šπ‘Žπ‘₯(π‘₯𝑖,𝑦𝑖) 𝑛 𝑖=1 π‘₯𝑖+𝑦𝑖 2 . (18) Due to recommendation in Garuti (2007), the values G < 0.9 correspond to incompatible vectors, otherwise the vectors are compatible. To check a precision of fit for the pairwise judgements by the priority vector estimate, we can use a definition of the elements ajk as quotients of preference between each pair of j- th and k-th items. With a vector-column of priority estimate alpha, we find its element- reciprocal vector-row (1/alpha)’ and build their outer product by the same pattern as used in Equation (2). With this outer product we find the quality of its fit for the matrix A Equation (1). The standard error (STE) is a measure of the mean distance between the observed and estimated pairwise ratios: 𝑆𝑇𝐸 = √ 1 𝑛2 βˆ‘ (π‘Žπ‘—π‘˜ βˆ’ 𝛼𝑗 π›Όπ‘˜ ) 2 𝑛 𝑗,π‘˜=1 . (19) IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 582 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 Another convenient measure of the precision for a matrix approximation by the vectors outer product is the mean absolute error (MAE): 𝑀𝐴𝐸 = 1 𝑛2 βˆ‘ |π‘Žπ‘—π‘˜ βˆ’ 𝛼𝑗 π›Όπ‘˜ |𝑛𝑗,π‘˜=1 . (20) The smaller the values of fit (Equations 19-20), the better the quality of the vector estimate. The measures of STE and MAE can be obtained by using Equations (19-20) only for the off-diagonal pairwise ratios equal or above 1 because they correspond to the elicited quotients of preference, and the reciprocal values below 1 are simply added at completion of the matrix (Equation 1) of pairwise judgements. Besides the characteristics of the residual mean values assessed via standard deviation (Equation 19) or absolute deviation (Equation 20), the quality of approximation of the pairwise judgements by the obtained priority vectors can be checked by a measure reminding the coefficient of multiple determination R 2 that is widely used in regression analysis. As shown in Lipovetsky (2009), this coefficient can be defined via the observed and estimated paired ratios of the priorities: 𝑅2 = 1 βˆ’ 𝑅𝑆𝑆 𝐸𝑆𝑆 = 1 βˆ’ βˆ‘ (π‘Žπ‘—π‘˜βˆ’ 𝛼𝑗 π›Όπ‘˜ ) 2 𝑛 𝑗,π‘˜=1 βˆ‘ (π‘Žπ‘—π‘˜βˆ’1) 2𝑛 𝑗,π‘˜=1 . (21) In the numerator (Equation 21) the residual sum of squares (RSS) of the estimated priority deviations from the elicited values is used, and the denominator is presented by the equivalent sum of squares (ESS) which assumes all the same preferences Ξ±j /Ξ±k ≑ 1. The coefficient (Equation 21) shows how much the found priorities outperform the case of absence of preferences among the alternatives. The better the approximation of the paired judgements by the estimated priorities is the closer the RSS is to zero, so the coefficient of determination R 2 is bigger and closer to one. In the absence of preferences Ξ±j/Ξ±k = 1, the numerator equals the denominator, and R 2 = 0. For the exact fit ajk = Ξ±j/Ξ±k for all judgements, RSS = 0, and R 2 = 1. The value R 2 commonly belongs to the interval from 0 to 1, which makes it a very convenient measure for comparison of the priority vectors obtained by different techniques. Only really poor estimates can produce a residual total RSS above the value of the equivalent residuals ESS, and it would be indicated by a negative R 2 . The characteristic (Equation 21) corresponds to the STE measure (Equation 19) of squared deviations, but it is possible to build the other estimates, for example, using the MAE residuals (Equation 20) as well. 4. Numerical comparisons for priority estimations Let us consider numerical examples of the priority estimations for three classical AHP problems. Example 1: the problem of β€œChoosing the best home” as described in Saaty and Kearns (1985), Saaty and Vargas (1994) and Saaty (1996). This matrix is also used for checking IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 583 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 some new approaches (Lipovetsky, 1996; Lipovetsky and Tishler, 1999; Lipovetsky and Conklin, 2002, 2015). The criteria of comparison are: 1 – size of house, 2 – location to bus, 3 – neighborhood, 4 – age of house, 5 - yard space, 6 – modern facilities, 7 – general condition, 8 – financing. The matrix of pairwise comparisons A (Equation 1) for this problem is presented in Table 1a. In this example with n=8, the maximum eigenvalue (Equation 3) of the matrix in Table 1a equals πœ† = 9.669. With the random consistency for this case RI=1.41, the consistency index and consistency ratio (Equations 14-15) are: 𝐢𝐼 = 9.669βˆ’8 7 = 0.238, 𝐢𝑅 = 0.238 1.41 = 0.169. A value of CR up to 10% is considered to indicate some inconsistency, so the obtained result of 17% is acceptable with a reservation, when the data could require a review of the elicited judgements, and in Lipovetsky and Conklin (2002) it was shown how to identify and to adjust the data in this case. Table 1a Example 1: Choosing the best home problem. Pairwise comparison matrix. item 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 1/5 1/3 1/7 1/6 1/6 3 4 5 1 3 1/5 1/3 1/3 5 7 3 1/3 1 1/6 1/3 1/4 1/6 5 7 5 6 1 3 4 7 8 6 3 3 1/3 1 2 5 6 6 3 4 1/4 1/2 1 5 6 1/3 1/5 6 1/7 1/5 1/5 1 2 1/4 1/7 1/5 1/8 1/6 1/6 1/2 1 Several methods of priority estimation for this data are presented in Table 1b. In its upper part, there are four estimates of the priority vector as follows: the classical EM solution (Equation 3), the LS estimation (Equation 4), the LN technique (Equation 5), and the robust estimation RE (Equation 13). All the vectors are normalized by the total of their elements equaling one (Equation 6). Judging by eye, all of the solutions are very similar by weight and the smallest and the biggest by importance are the age and financing of the house, items 4 and 8, respectively. For comparison between the obtained four priority vectors we applied the measures (Equations 16-18) of correlations, S-compatibility, and G- compatibility, presented after the vectors in the three matrices in Table 1b. Judging by the correlations, all of the vectors are close enough by their structure and the LS is a bit further from the other three. The measure of S-compatibility proves that the EM, LN, and RE vectors are similar, within less than the required threshold of 10% of the S-index deviation from one. The more sensitive G-compatibility demonstrates that the pair of EM and LN vectors are close with G=0.927, and the two vectors LN and RE are close with G=0.912, which are IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 584 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 values above the threshold 0.9 needed for viewing the corresponding vectors as compatible. Table 1b Example 1: Choosing the best home problem. Priority vector estimations Item EM LS LN RE 1. size of house 0.173 0.199 0.175 0.150 2. location to bus 0.054 0.100 0.063 0.054 3. neighborhood 0.188 0.148 0.149 0.141 4. age of house 0.018 0.017 0.019 0.022 5. yard space 0.031 0.045 0.036 0.037 6. modern facilities 0.036 0.065 0.042 0.041 7. general condition 0.167 0.184 0.167 0.163 8. financing 0.333 0.242 0.350 0.392 Correlations EM 1 0.935 0.988 0.972 LS 0.935 1 0.933 0.881 LN 0.988 0.933 1 0.991 RE 0.972 0.881 0.991 1 S-compatibility EM 1 1.113 1.015 1.028 LS 1.113 1 1.071 1.122 LN 1.015 1.071 1 1.010 RE 1.028 1.122 1.010 1 G-compatibility EM 1 0.774 0.927 0.865 LS 0.774 1 0.809 0.742 LN 0.927 0.809 1 0.912 RE 0.865 0.742 0.912 1 Precision STE 2.071 1.849 1.813 1.831 MAE 1.079 1.083 0.958 0.934 R 2 0.423 0.540 0.558 0.549 The last segment at the bottom of Table 1b displays the precision by Equations (19-21) for each vector solution. By the minimum standard error STE, the best model is the LN, and by the mean absolute error MAE, the best model is the RE. The values of the MAE also suggest that an average deviation from the observed pair judgements evaluated by the obtained quotients from a priority vector is not more than one unit. The coefficient of multiple determination R 2 in the last row of Table 1b shows by its maximum values that the LN and RE models outperform the other two models, though all R 2 values are not IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 585 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 high, which indicates a difficulty in approximation of inconsistent judgements by a priority vector in any estimation. Example 2: the problem of β€œDistance from Philadelphia” is one of the first AHP problems described by Saaty (1977). The distance of six cities from Philadelphia was estimated by the criterion: for each pair of cities, how many times farther is the more distant city located from Philadelphia than the nearer one? The elicited data is presented in Table 2a. Table 2a Example 2: Distance from Philadelphia problem. Pairwise comparison matrix Airport CAI TYO ORD SFO LGW YMX Cairo.CAI 1 0.333 8 3 3 7 Tokyo.TYO 3 1 9 3 3 9 Chicago.ORD 0.125 0.111 1 0.167 0.2 2 SanFrancisco.SFO 0.333 0.333 6 1 0.333 6 London.LGW 0.333 0.333 5 3 1 6 Montreal.YMX 0.143 0.111 0.5 0.167 0.167 1 The maximum eigenvalue (Equation 3) in this example equals πœ† = 6.454. The random consistency for n=6 is RI=1.24, then the consistency index and consistency ratio Equations (14-15) are: 𝐢𝐼 = 6.454βˆ’6 5 = 0.091, 𝐢𝑅 = 0.091 1.24 = 0.073 . The value of CR=7.3% is less than the 10% permitted, which allows one to conclude that the data on pair judgements is sufficiently consistent. Table 2b presents the results of priority estimations for this example and is organized as the previous Table 1b, but with one additional column of the actual shares of distances known in this case. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 586 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 Table 2b Example 2: Distance from Philadelphia problem. Priority vector estimations City EM LS LN RE actual 1. Cairo 0.262 0.254 0.260 0.239 0.278 2. Tokyo 0.397 0.305 0.399 0.447 0.361 3. Chicago 0.033 0.047 0.035 0.034 0.032 4. San Francisco 0.116 0.186 0.116 0.104 0.132 5. London 0.164 0.184 0.163 0.147 0.177 6. Montreal 0.027 0.024 0.027 0.029 0.019 Correlations EM 1 0.943 1.000 0.990 0.991 LS 0.943 1 0.941 0.898 0.973 LN 1.000 0.941 1 0.991 0.990 RE 0.990 0.898 0.991 1 0.962 Actual 0.991 0.973 0.990 0.962 1 S-compatibility EM 1 1.064 1.000 1.009 1.024 LS 1.064 1 1.064 1.106 1.045 LN 1.000 1.064 1 1.008 1.027 RE 1.009 1.106 1.008 1 1.060 Actual 1.024 1.045 1.027 1.060 1 G-compatibility EM 1 0.821 0.993 0.900 0.914 LS 0.821 1 0.820 0.753 0.854 LN 0.993 0.820 1 0.905 0.908 RE 0.900 0.753 0.905 1 0.823 Actual 0.914 0.854 0.908 0.823 1 Precision STE 1.390 1.340 1.333 1.523 2.295 MAE 0.696 0.862 0.686 0.790 1.012 R 2 0.794 0.809 0.810 0.753 0.438 We see that in general the vectors are similar and each one makes sense as proportionally scaled distances from Philadelphia to other cities in the USA, as well as to other countries and continents. The pair correlations also show that the vectors are closely related to the actual distances, and the same is supported by the S-compatibility index. G-compatibility indicates that the EM and LN vectors are compatible with the actual shares of distances. The precision of the reproduction of the judgement matrix is high, especially with the LS and LN methods. The precision measured by STE, MAE, and R 2 of the actual distances is the worst one within the other values in the last rows of Table 2b. This means that the pair judgements on distances correspond to the priority vectors rather than to the actual distance shares. Therefore, in this data we do not need to use the actual data to consider compatibility among the vectors. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 587 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 Example 3. The data for this problem is given in Whitaker (2007) where the area of five geometric figures was compared – see the matrix of pair judgements in Table 3a. Table 3a Example 3: Geometric figures’ area problem. Pairwise comparison matrix Figure Circle Triangle Square Diamond Rectangle Circle 1 9 2.5 3 6 Triangle 0.111 1 0.2 0.286 0.667 Square 0.4 5 1 1.7 3 Diamond 0.333 3.5 0.588 1 1.5 Rectangle 0.167 1.5 0.333 0.667 1 The maximum eigenvalue of this matrix is πœ† = 5.026. The random consistency for n=5 is RI=1.12, so the consistency index and consistency ratio (Equations 14-15) equal the following values: 𝐢𝐼 = 5.026βˆ’5 4 = 0.006, 𝐢𝑅 = 0.006 1.12 = 0.006 . CR=0.6% which proves a very high level of consistency of this data. This can be explained by the pairwise ratios that were used where not only the integer numbers but also the rational numbers (like 2.5 or 3.5) were permitted in the preference evaluation. Table 3b presents the priority estimation results for this example, and is organized as Table 2b, with the additional column of the actual shares of the areas measured for these figures. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 588 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 Table 3b Example 3: Geometric figures’ area problem. Priority vector estimations EM LS LN RE actual 1. Circle 0.488 0.464 0.487 0.496 0.470 2. Triangle 0.049 0.050 0.049 0.049 0.050 3. Square 0.233 0.248 0.233 0.225 0.240 4. Diamond 0.148 0.159 0.148 0.147 0.140 5. Rectangle 0.082 0.078 0.082 0.083 0.090 Correlations EM 1 0.998 0.999 0.999 0.999 LS 0.998 1 0.998 0.995 0.998 LN 0.999 0.998 1 0.999 0.999 RE 0.999 0.995 0.999 1 0.998 actual 0.999 0.998 0.999 0.998 1 S-compatibility EM 1 1.003 1.000 1.000 1.003 LS 1.003 1 1.003 1.004 1.007 LN 1.000 1.003 1 1.000 1.003 RE 1.000 1.004 1.000 1 1.003 actual 1.003 1.007 1.003 1.003 1 G-compatibility EM 1 0.948 0.999 0.982 0.955 LS 0.948 1 0.948 0.931 0.953 LN 0.999 0.948 1 0.982 0.956 RE 0.982 0.931 0.982 1 0.940 actual 0.955 0.953 0.956 0.940 1 Precision STE 0.253 0.195 0.253 0.289 0.279 MAE 0.145 0.110 0.145 0.162 0.165 R 2 0.987 0.992 0.987 0.983 0.985 All the vector estimates in this data practically coincide; the pair correlations are very high, and both the S- and G- indices prove compatibility among the estimates and with the actual observations. The precision measured by the STE, MAE, and R 2 characteristics demonstrates a high quality of fit of the data by any of the estimated vectors of priority and by the actual values as well. 5. Conclusions The paper considered several methods of priority vector evaluation in the AHP. They include the classical eigenproblem method, least squares, multiplicative or logarithmic approach, and a robust estimation based on transformation of the pairwise ratios to the shares of preferences. Together with estimation of the vectors, validation of data IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 589 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 consistency and comparison of vectors by correlations, the S- and G- compatibility indices were also completed. The numerical results for different data sizes and consistency indices demonstrate that all of the methods produce compatible results for the consistent data, otherwise a discrepancy between the different methods of the priority estimation would be observed. Therefore, the data consistency should always be proved before the vector evaluation. Another important conclusion concerns the precision assessment for the data matrix approximation by the obtained priority vectors. Any regular statistical modeling requires a verification of the produced results by some quality characteristics. For example, in regression analysis, measures like the residual standard error STE, mean absolute error MAE, and coefficient of multiple determination R 2 are commonly employed. Applying them in the AHP environment can enrich the evaluation and interpretation of the results on priority modeling and is demonstrated on the numerical estimations performed in the paper. For instance, in the data for Example 1 with a low consistency, the R 2 values are also not high which indicates a difficulty of approximation of inconsistent judgements by a priority vector in any estimation, and by MAE values a mean deviation of the quotients of a priority vector’s elements from the observed pair judgements could be as big as one unit. In Example 2 with a good consistency, the precision of the reproduction of the judgement matrix by the found priority vectors is high enough, although at the same time the actual distances occurred to yield the worst vector for approximation of the elicited pairwise priority matrix Therefore, in this data we should not use the actual data on distances to check the compatibility with the obtained estimates of the vectors. Example 3 with a perfect consistency yields all vectors of high compatibility and of a great quality of the elicited judgements reconstruction by each vector’s quotients of preference. The considered methods of priority vector estimation and characteristics of their quality are convenient and helpful in practical applications of the AHP for solving various multi- criteria decision making problems. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 590 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 REFERENCES Garuti A. C. (2007). Measuring compatibility (closeness) in weighted environments., Proceedings of the International Symposium on the AHP, Vina del Mar, Chile, August 2– 6. Garuti A. C. and Salomon V.A.P. (2011). Compatibility indices between priority vectors. International Journal of the Analytic Hierarchy Process, 4(2), 152-160. Doi: https://doi.org/10.13033/ijahp.v4i2.130 Lipovetsky S. (1996). The synthetic hierarchy method: an optimizing approach to obtaining priorities in the AHP. European Journal of Operational Research, 93, 550-564. Doi: https://doi.org/10.1016/0377-2217(95)00085-2 Lipovetsky S. (2009) Global priority estimation in multiperson decision making. Journal of Optimization Theory and Applications, 140, 77-91. Doi: https://doi.org/10.1007/s10957-008- 9447-6 Lipovetsky S. (2013). Supermatrix’ eigenproblem and interpretation of priority vectors in Analytic Network Process. International Journal of the Analytic Hierarchy Process, 5(1), 105-113. Doi: https://doi.org/10.13033/ijahp.v5i1.132 Lipovetsky S. and Conklin W.M. (2002). Robust estimation of priorities in the AHP. European Journal of Operational Research, 137, 110-122. Doi: https://doi.org/10.1016/s0377-2217(01)00071-6 Lipovetsky S. and Conklin M. (2015). AHP priorities and Markov-Chapman- Kolmogorov steady-states probabilities. International Journal of the Analytic Hierarchy Process, 7(2), 2015, 349-363. Doi: https://doi.org/10.13033/ijahp.v7i2.243 Lipovetsky S. and Tishler A. (1999). Interval estimation of priorities in the AHP. European Journal of Operational Research, 114, 153-164. Doi: https://doi.org/10.1016/s0377-2217(98)00012-5 Lootsma F. (1993) Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 2, 87-110. Doi: https://doi.org/10.1002/mcda.4020020205 Lootsma F. (1999). Multi-Criteria Decision Analysis via ratio and difference judgement. Dordrecht, Boston, London: Kluwer Academic Publishers. Doi: https://doi.org/10.1007/b102374 Saaty T.L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15, 234–281. Doi: https://doi.org/10.1016/0022- 2496(77)90033-5 Saaty T.L. (1980). The Analytic Hierarchy Process. New York: McGraw-Hill. IJAHP Article: Lipovetsky/Priority vector estimation: consistency, compatibility, precision International Journal of the Analytic Hierarchy Process 591 Vol. 12 Issue 3 2020 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 Saaty T.L. (1994) Fundamentals of decision making and priority theory with the Analytic Hierarchy Process. Pittsburgh, PA: RWS Publications. Saaty T.L. (1996) Decision making with dependence and feedback: The Analytic Network Process. Pittsburgh, PA: RWS Publications. Saaty T. L. (2005) Theory and applications of the Analytic Network Process: decision making with benefits, opportunities, costs, and risks. Pittsburgh, PA: RWS Publications. Saaty T.L. and Kearns K.P. (1985). Analytical planning. New York: Pergamon Press, New York. Saaty T.L. and Peniwati K. (2007). Group decision-making: Drawing out and reconciling differences. Pittsburgh, PA: RWS Publications. Saaty T.L. and Vargas L.G. (1984). Comparison of eigenvalue, logarithmic least squares and least squares methods in estimating ratios. Mathematical Modelling, 5, 309-324. Doi: https://doi.org/10.1016/0270-0255(84)90008-3 Saaty T.L. and Vargas L.G. (1994). Decision making in economic, political, social and technological environment with the Analytic Hierarchy Process. Pittsburgh, PA: RWS Publications, Pittsburgh, PA. Whitaker R. (2007). Validation examples of the Analytic Hierarchy Process and Analytic Network Process. Mathematical and Computer Modelling, 46, 840–859. Doi: https://doi.org/10.1016/j.mcm.2007.03.018