IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 1 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 ANALYZING THE AHP PRIORITY VECTORS: GOING BEYOND INCONSISTENCY INDEXES Marcelo Neto Botelho Instituto de Educação Tecnológica Brazil marcelo.botelho1@hotmail.com ABSTRACT A great concern when utilizing the Analytic Hierarchy Process (AHP) is how the final priority vector, resulting from the inconsistent analysis, behaves when compared with the original priority ranking, resulting from the consistent analysis. The AHP utilizes an inconsistency index to predict rank reversal. In addition to the original inconsistency index of the AHP, several authors have worked on developing alternative inconsistency indexes, with the goal of improving the predictability of rank reversal. However, inconsistency indexes do not help clarify whether a rank reversal is a rejectable outcome or, to some extent, the correct answer. A rank reversal may express the correct priority, particularly when some positions in the original priority rank have small weight differences among them. Therefore, it is very important to develop a method to allow a clear and definitive analysis on how disturbed the weights and ranking of the final priority vectors are when compared to their original consistent rankings. Such a method is developed here and its utilization is demonstrated by analyzing a corporate governance scenario. Keywords: AHP; MCDA; rank reversal; inconsistency index; corporate governance 1. Introduction The AHP works to support decisions as different criteria and alternatives are considered. These criteria and alternatives can combine objective and subjective parameters. The AHP, as proposed by Saaty (1977), generates a priority vector from a Pairwise Comparison Matrix (PCM). Such pairwise comparison, made by the Decision Makers (DM) involved in the respective analysis, carries a certain level of inconsistency derived from the inherent subjectivity of human scrutiny. This ability of the AHP to absorb a certain level of inconsistency is highly valuable. Saaty (1997) emphasizes that the AHP places special focus on integrating human judgment into decision-making process and on evaluating the consistency of such judgments. Saaty (1977) also comments that a certain level of inconsistency in a PCM does not necessarily affect the ranking of the final priority vector. Grybowski and Starczewski (2020) introduced the SWFR – significantly-wrong-final- ranking – as a concept to analyze whether a certain rank reversal should represent the rejection of the final priority vector (FPV). Buede and Maxwell (1995) state that “Through formal and informal discussions about rank reversal, the focus has been on mailto:marcelo.botelho1@hotmail.com IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 2 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 assessing whether the problem exists, what the reasons are, as well as whether the rank reversal is the problem or the desired response”. Wang and Triantaphyllou (2008) investigated the rank reversal that occurred in a case of MCDA, not only analyzing the inconsistency index, but analyzing the changes that actually occurred in the ranking positions. Different authors have worked on the development of other inconsistency indexes besides the one originally proposed by Saaty (2009). Saaty’s inconsistency index is defined as 𝐼𝐶 = (λ𝑚𝑎𝑥 − 𝑛)/(𝑛 − 1) where λmax is the maximum eigenvalue of the PCM. Bozóki and Rapcsák (2015) presented alternative inconsistency indexes that have their particular threshold of acceptability, while Grzybowski and Starczewski (2020) introduced the so-called IC ATIA. However, although these are serious contributions to the assessment of the inconsistency in a pairwise comparison, these indexes do not contribute to the assessment of the real impact on final ranking positions. Furthermore, it is important to note that even in the case of no rank reversal, an investigation into the disturbance that occurred in the FPV may be of interest. Cases, for example, that involve the allocation of resources among different alternatives may imply a deeper evaluation on the weight distribution along the ranking positions, even if no reversal has been observed. Finally, a certain magnitude of rank reversal can be accepted, as it can represent proper prioritization analysis. For example, this occurs in cases where the DM made some pairwise comparisons considering a very similar level of relevance (in the limit, a tie). In these cases, the analysis of inconsistencies can lead to some sorting reversal that, in fact, is adequately clarifying and segregating the criteria judgement performed by the DMs. These considerations support the need to establish a methodology that goes beyond inconsistency indexes to allow an easy, clear, complete and definitive analysis of the FPV generated via the AHP. To test the method and metrics developed here to evaluate the FPV, an application in a corporate governance scenario is analyzed. A governance maturity analysis has been chosen since, generally, the relevance of the governance criteria do not differ that much. The subjectivity of governance issues is highlighted by IBGC (2015) 1 “In the exercise of corporate governance, the topics dealt with are often subjective and ambiguous, which requires from the governance agents the ability to assess, reason and judge”. The development of the proposed methods and the results regarding the prioritization of the governance theme are presented in this study. The analysis of governance issues, with the support of the methods developed here, also compared the use of the original scale by Saaty (OSS) (1997) and the Generalized Balanced Scale (GBS), as proposed by Goepel (2018). 2. Technical background 2.1 The original AHP methodology The AHP considers the Perron-Frobenius theory, where the maximum autovector (maximum eigenvector) of a matrix containing positive values forms the priority vector. Furthemore, Saaty (1977) stipulates that a pairwise comparison is the best 1 IBGC: Brazilian Institute of Corporate Governance. IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 3 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 method that humans can use to compare different criteria or alternatives. Saaty (1977) refers to the studies by Weber (1846) and Fechner (1860), which focus on the evaluation of human response to stimuli, to define the scale adopted in the original AHP methodology. Saaty (1977) defines a scale from 1 to 9 (Original Saaty Scale – OSS), linked to nine levels of criteria relevance. The pairwise comparison can introduce an inconsistency into the Pairwise Comparison Matrix (PCM). This inconsistency is a result of the fact that DMs may not maintain perfect proportionality among their comparisons in their pairwise comparisons. When the comparison maintains proper proportionality, there will be a consistent PCM and when not, an inconsistent one. In principle, the inconsistency embodies the subjectivity of human judgements. Due to such inconsistencies, Saaty (1977) defined the Consistency Ratio (CR) with a threshold of 10% 2 as the limit for accepting the AHP autovector. The figures below summarize the main steps and concepts of the AHP methodology. Figure 1 Original Saaty Scale (OSS) Figure 2 Pairwise Comparison Matrix (PCM) The Final Priority Vector (FPV) w(i,j) calculated in the AHP method is defined as: [w1,j] = [Ai,j] x [C1,j], where: [w1,j] = Final Priority Vector (FPV); [Ai,j] = Matrix of normalized priority vectors of the alternatives [C1,j] = Criteria eigenvector and the Saaty inconsistency index is defined as 𝐶𝐼 = ((λ𝑚𝑎𝑥 − 𝑛))/((𝑛 − 1)), where λmax = PCM Autovalue and n = number of criteria. 2 Consistency Ratio: the relation between CI and RI. RI is the random consistency index as per Saaty (1987) 1 2 3 4 5 6 7 8 9 Equal Importance Intermediate Moderate Importance Intermediate Strong Importance Intermediate Very Strong Importance Intermediate Extreme Importance C1 C2 C3 C4 ....... Cn C1 1,0 (P1/p1) P2/P1 P3/P1 P4/P1 ....... Pf/P1 C2 P1/P2 1,0 (P2/p2) P3/P2 P4/P2 ....... Pf/P2 C3 P1/P3 P2/P3 1,0 (P3/p3) P4/P3 ....... Pf/P3 C4 P1/P4 P2/P4 P3/P4 1,0 (P4/p4) ...... Pf/P4 ....... ....... ....... ....... ....... ....... ...... Cn P1/Pn P2/Pn P3/Pn P4/Pn ...... 1,0 (Pf/Pn) Pairwise Comparison Matrix - PCM Parameters (p1 to Pf = weights 1 to 9) of Criteria (C1 to Cn) Diagonal = 1,0 once a Criteria is compared with itself First Line = Consistent Comparisons (All Criterias compared with first one) Region of potential Inconsistencies (comparisons not obligatorily proportional to Fisrt Line ) Region of reciprocal comparisons IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 4 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 2.2 A case study: corporate governance maturity assessment To test the method and metrics developed here to assess the FPV, an application in a corporate governance scenario was analyzed. To this end, a methodology was structured to assess the maturity of governance based on the proposal by Álvares, Giacometti and Gusso (2008). Botelho (2021) presents more details about the choice of this methodology and the tool that was developed to assess the maturity of corporate governance. The evolution of each element is fictitious (ranging from 30% to 100%) and the pairwise comparison of the criteria involved in this governance maturity was obtained from directors invited from seven different companies. The AHP methodology was inserted into the aforementioned governance analysis tool, generating an agenda where, taking into account the pairwise criteria comparison provided by the seven board members, the elements' priorities (the alternatives) were prioritized. Therefore, this exercise allowed a comparison between the original ranking (derived from consistent PCM) and the FPV. In addition, a comparison was also made between the FPV and the priority agenda organized via the AHP. Last but not least, both sets of comparisons were performed using OSS and GBS balances. Figure 3 presents a summary of the mentioned tool, and Figure 4 shows an example of a priority agenda generated by the AHP. Figure 3 Hierarchy analysis of governance maturity CRITERIAS -> DIMENSIONS -> 1.1.1 - Values 67% 2.1.1- Corporate 50% 2.2.1- Stock exchange 83% 3.1.1- Right to vote and oversight 33% 3.2.1- Responsibility to employees 90% 4.1.1- Adm. Council (Board) 66,7% 4.2.1- Adm. Council (Board) 83% 4.3.1- Member assembly 100% 5.1.1- CEO Hiring and Succession 67% 5.2.1- Capital structure 50% 1.1.2- Mission and Vision 88% 2.1.2- Tax and social security 50% 2.2.2- SEC & SOX 38% 3.1.2- Profit sharing 50% 3.2.2- Responsibility to the market 40% 4.1.2- Committe es 30,0% 4.2.2- Committees 30% 4.3.2- Shareholde r's assembly 92% 5.1.2- CEO performance and compensation assessment 100% 5.2.2- Conflict resolution mechanism 100% 1.1.3- Code of ethics 50% 2.1.3- Labor laws 100% 2.2.3- Ibovespa 30% 3.1.3- Leave the Organization 67% 3.2.3- Responsibility to society 33% 4.1.3- Fiscal Committe 41,7% 4.2.3- Fiscal Committe 100% 5.1.3- Board of Directors evaluation 67% 5.2.3- Functional Policies 83% 1.1.4- Shareholders Agreeement 83% 3.2.4- Responsibility to the 88% 4.1.4- Manag. Director 50,0% 4.2.4- Manag. Director 100% 5.2.4- Guidelines and 100% 3.2.5- Work Safety and Occupational Health 88% 4.1.5- Holding 37,5% 4.2.5- Holding 50% 5.2.5- Strategic management 100% 4.1.6- Independ ent auditing council 40,0% 4.2.6- Independen t auditing council 50% 5.2.6- Market relations 50% 5.2.7- Risk management 30% 5.2- Ownership (shareholders / partners) / BOARD / CEO 1 -Alignment Governance Maturity ELEMENTS -> 5- Processes4- Structure3- Sustainability2- Conformity 1.1- Corporate and business cohesion 2.2- Regulatory Requirements 2.1- Legal Conformities 3.1- Responsibility to partners 3.2- Responsibility to stakeholders 4.1- Organs 4.2- Roles and Effectiveness 4.3- Operation and procedures 5.1- Administ. Council IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 5 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Figure 4 Priority agenda (via AHP over the hierarchy analysis of governance maturity) 2.3 An alternative scale Several authors have worked on other scales of development besides the OSS. The goal with these alternative scales is to improve the predictability of rank reversal. This study compares the use of the OSS and an alternative scale, an AHP applied process of prioritizing corporate governance criteria, as presented by Botelho (2021). Goepel (2018) compared different weight scales and introduced the Generalized Balanced Scale (GBS). Goepel (2018) demonstrated that GBS performed better reaching small weight uncertainty and weight dispersion compared to the others. Therefore, Botelho (2021) and this study adopted the GBS as the alternative scale to be applied in the aforementioned AHP governance prioritizing exercise. The GBS is defined as: where c = weight value in GBS; n = number of criteria and x = weight value in OSS. Figure 5 shows GBS weights compared with OSS. Figure 5 OSS & GBS OSS GBS n=3 GBS n=4 GBS n=5 GBS n=6 GBS n=7 1 1,0 1,0 1,0 1,0 1,0 2 1,3 1,4 1,4 1,5 1,5 3 1,7 1,8 1,9 2,0 2,1 4 2,1 2,3 2,5 2,6 2,8 5 2,7 3,0 3,2 3,4 3,5 6 3,5 3,9 4,1 4,3 4,5 7 4,6 5,0 5,3 5,5 5,7 8 6,3 6,6 6,8 7,0 7,1 9 9,0 9,0 9,0 9,0 9,0 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 6 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 3. Methodology This study can be classified as applied research, as it aims to generate knowledge for the practical application of scenario analysis. In addition, it uses a practical case study to address the proposal. The methodology developed here, as mentioned in the introduction, presents a complete criterion for evaluating the priority vectors generated by AHP independently and not limited to the consistency indices. Below, the development of this new approach is demonstrated with respect to a complete analysis of the level of perturbation suffered by the FPVs when compared to the original ordering vectors. 3.1 Analyzing final priority vectors without rank reversal occurrences Given an Initial Priority Vector (IPV) [w(i)1, w(i)2, w(i)3, .... w(i) [n-1], w(i)n] (the eigenvector of the consistent PCM) and a final one (FPV) [w(f)1, w(f)2, w(f)3,.... w(f)[n-1], w(f)n] (the eigenvector of the inconsistent PCM), the absolute difference between the weights (wf - wi), taken in inverse proportion to their positions in the ranking, weights the intensity of the disturbance, providing a No Reversion Index (NRI), as follows: NRI= ∑ [𝑤𝑖(𝑖) − 𝑤𝑖(𝑓)]/𝑖 𝑛𝑖=1 (1) See Figure 6 for a NRI definition and Figures 7 and 8 for a NRI histogram and NRI cumulative frequency, respectively. Furthermore, it established a metric to qualify the level of disturbance without rank reversal. For that, a Monte Carlo analysis was performed evaluating 13.200 random simulations without rank reversal. This Monte Carlo exercise considered a vector with 5 positions in the ranking (n=5). It has defined the “No Reversal Quality Vector” (NRQV), according to Equation 2. Figure 6 Analysis without rank reversal IPV (Consistent) FPV (Inconsistent) wi (1) wf(1) [wi (1) - wf(1)] / 1 wi (2) wf(2) [wi (2) - wf(2)] / 2 wi (3) wf(3) [wi (3) - wf(3)] / 3 . . . . . . . . . wi (n-2) wf(n-2) [wi (n-2) - wf(n-2)] / (n-2) wi (n-1) wf(n-1) [wi (n-1) - wf(n-1)] /(n-1) wi (n) wf(n) [wi (n) - wf(n)] / n IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 7 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Figure 7 NRI histogram Figure 8 NRI’s cumulative frequency Therefore, the IPV disturbance level is reflected via the NRI index and classified by the NRQV index, represented in Equation 2 below, in line with Figure 8 (for n=5). 𝑁𝑅𝑄𝑉(5) = −632. 𝑁𝑅𝐼6 + 1473. 𝑁𝑅𝐼5 − 1307. 𝑁𝑅𝐼4 + 538. 𝑁𝑅𝐼3 − 93. 𝑁𝑅𝐼2 + 𝑁𝑅𝐼 + 1 (2) 3.2 Analyzing final priority vectors with rank reversal occurrences Similar to the assessment without rank reversal, the impact on the FPV when a rank reversal occurs is more significant the higher in these rankings the positions affected by the reversal are. The higher the original ranking position impacted by the reversal, the higher the level of relevance of this disturbance. Therefore, an index to represent the magnitude of this particular case of reversal must consider the impact inversely proportional to the original ranking position. IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 8 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Considering that, an Index of Reversal (IR) is defined as: IR = ∑ [𝑃𝑖(𝑖) − 𝑃𝑖(𝑓)]/𝑖 𝑛𝑖=1 (3) where: Pi(i) = Position “i” in original ranking e Pf(i)= Position “i” in final ranking In addition to the IR index, a complementary metric is needed to evaluate how significant the reversal was. To determine this, a minimum and a maximum reversal impact index must be defined. The minimum reversal that can be observed is between the penultimate (n-1) and the last position (n) and only a draw between these positions (n and n-1) represents a lighter impact. This observation is taken to define the so-called Index of Minimum Reversal (IMR) (see Equations 4 and 5). 𝐼𝑀𝑅 (𝑟𝑒𝑣𝑒𝑟𝑠𝑎𝑙) = [(1/𝑛) + 1/(𝑛 − 1)]  𝐼𝑀𝑅(𝑟𝑒𝑣𝑒𝑟𝑠𝑎𝑙) = [(2𝑛 − 1)/(𝑛(𝑛 − 1))] (4) 𝐼𝑀𝑅 (𝑑𝑟𝑎𝑤) = [(𝑛 − (𝑛 − 1))/(𝑛 − 1)]  𝐼𝑀𝑅 (𝑑𝑟𝑎𝑤) = [1/(𝑛 − 1)] (5) Table 1 IMR x n (reversal and draw on last ranking positions) Regarding the definition of a maximum reversal impact index, an extremely relevant ranking reversal that can be conceived is the complete symmetric reversal of the ranking (the first position is reversed with the last, the second with the penultimate, the third with the antepenultimate, successively). It is a fact that significant reversals are undesirable. Therefore, to establish a Maximum Reversal Impact Index (MRII) this must be taken into account and a Monte Carlo simulation carried out to verify the level of confidence of this assumption. Figure 9 presents the above concept, and Figure 10 shows confidence level check. n 2 3 4 5 6 7 8 9 10 IMR - Reversal of n and (n-1) . 1.50 0.83 0.58 0.45 0.37 0.31 0.27 0.24 0.21 IMR - Draw between (n) and (n-1) 1.00 0.50 0.33 0.25 0.20 0.17 0.14 0.13 0.11 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 9 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 𝑈𝑝𝑤𝑎𝑟𝑑𝑠 = (𝑛 − 1)/1 + (𝑛 − 3)/2 + (𝑛 − 5)/3 + (𝑛 − 7)/4 + ⋯ Therefore, 𝑈𝑝𝑤𝑎𝑟𝑑𝑠 = ∑ (𝑛 + 1 − 2𝑖)/𝑖 𝑖𝑛𝑡( 𝑛 2 ) 𝑖=1 𝐷𝑜𝑤𝑛𝑤𝑎𝑟𝑑𝑠 = 𝑛 − 1 𝑛 + 𝑛 − 3 𝑛 − 1 + 𝑛 − 5 𝑛 − 2 + 𝑛 − 7 𝑛 − 3 + ⋯ Therefore, 𝐷𝑜𝑤𝑛𝑤𝑎𝑟𝑑𝑠 = ∑ (𝑛 + 1 − 2𝑖)/(𝑛 − 𝑖 + 1) 𝑖𝑛𝑡( 𝑛 2 ) 𝑖=1 Figure 9 Basis for MRII definition Therefore, in total: 𝑀𝑅𝐼𝐼 = ∑ (𝑛 + 1 − 2𝑖). ( 1 𝑖 + 1 𝑛−𝑖+1 ) 𝑖𝑛𝑡( 𝑛 2 ) 𝑖=1 (6) Table 2 MRII MRII 0.00 1.50 2.67 4.58 6.30 8.52 10.59 12.84 15.38 18.01 n 1 2 3 4 5 6 7 8 9 10 1 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 2 0.50 1.00 1.50 2.00 2.50 3.00 3.50 3 0.33 0.67 1.00 1.33 1.67 4 0.25 0.50 0.75 5 0.20 6 0.17 7 0.33 0.43 8 0.25 0.40 0.50 0.57 0.63 9 0.33 0.50 0.60 0.67 0.71 0.75 0.78 10 0.50 0.67 0.75 0.80 0.83 0.86 0.88 0.89 0.90 Dsc Asc IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 10 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Figure 10 MRII (n=5) level of confidence 3 Figure 10 shows a confidence level of 97.3% for the adopted MRII concept. Therefore, based on the above methodologies, an index to qualify the Reversal Quality Vector (RQV), with regard to the relevance of FPV rank reversal, is defined as: 𝑅𝑄𝑉 = 1 − [ 𝐼𝑅−𝐼𝑀𝑅 𝑀𝑅𝐼𝐼−𝐼𝑀𝑅 ]𝑘 (7) Since the ranking reversal of higher magnitudes is not desired, the RVQ was defined considering the exponential characteristic, with the exponent K of ¼ to induce significant reductions in this quality index. Thus, using this methodology, the governance scenarios were analyzed. These comparisons were considered in a theoretical status of governance maturity. Two different AHP scales (OSS and GBS) were applied. Finally, a comparison was also made between IPV (from consistent PCMs) and FPV (from inconsistent PCMs), as well as between FPV and the priority agenda of elements of governance. The results and discussions are presented in the next sections. 3 19,842 occurrences with rank reverse obtained from 20,000 simulations where, randomly, normalized vectors with 5 positions (n=5) had their initial and final ranking compared. IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 11 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 4. Results Table 3 Summary of indexes considering 7 pairwise comparison (between consistent and inconsistent PCMs) 4.1 Analysis of FPV x IPV (PCM Consistent x Inconsistent): Table 4 NRI Table 5 NRQV Board Member ESCALAS ISR IR QSR QCR IC Saaty 01 ~09 0.057 1.50 0.921 0.326 EGB 0.038 1.50 0.987 0.326 01 ~09 0.046 0.00 0.935 1.000 EGB 0.031 0.00 0.991 1.000 01 ~09 0.038 0.00 0.967 1.000 EGB 0.023 0.00 0.977 1.000 01 ~09 0.0163 0.00 0.9837 1.0000 EGB 0.0111 0.00 0.9889 1.0000 01 ~09 0.0383 0.33 0.9726 0.6574 EGB 0.0285 0.87 0.9752 0.4350 01 ~09 0.0258 0.00 0.9742 1.0000 EGB 0.0047 0.00 0.9953 1.0000 01 ~09 0.0436 0.25 0.9501 1.0000 EGB 0.0328 0.25 0.9894 1.0000 C7(*) 0.00% 4.92% PCM Inconsistent x Consistent 16.97% 11.73% (*): Draw in some positions of IPV (out of Consistents PCMs) C6 8.04% 6.34% C1 (*) C5 (*) 15.92% C2 C3 C4 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 12 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Table 6 IR Table 7 RQV 4.2 Analysis of FPV x priority agenda of governance maturity: Table 8 RQV (Inconsistent PCM) Table 9 RQV (Consistent PCM) 4.3 Interpretation of results From the above tables, it can be noted that: a) Concerning the comparisons between inconsistent and consistent PCMs: i. The GBS, in all cases, generated a lower level of disturbance in the FPV when no rank reversal has occurred. ii. When rank reversal occurred, the GBS generated a higher disturbance in the FPV than OSS. b) Concerning the comparisons between FPV and the priority governance agenda: i. FPV of inconsistent PCMs presented a higher magnitude of rank reversal with GBS than OSS. IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 13 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 ii. No difference was observed regarding the consistent PCMs iii. The average level of disturbance due to rank reversal was higher with the consistent PCM than with inconsistent ones, for both weight scales (average RQV of 0.61 for GBS and 0.87 for OSS compared to 0.58 for consistent PCMs). 5. Conclusion Based on these interpretations of the results, it can be concluded that: I. GBS is a more sensitive scale than OSS, as far as rank reversal trends are concerned. II. GBS generates a smaller weight difference among ranking positions. Therefore, if the perturbation is not enough to generate rank reversal, GBS provides a FPV with lower weight perturbations. III. The inconsistent analysis of the invited Board Members generated FPVs with higher adherence to the governance priority agenda than the consistent one. The inconsistency introduced by the pairwise comparison provided by the Board members is an indication that, indeed, such inconsistencies were a refinement/optimization of the process. Therefore, it reinforces the AHP methodology as an adequate process for decision making involving corporate governance issues. IV. The metrics developed in this study allowed a deeper understanding of the disturbance that occurred with FPV. These disturbances could not be quantified if only the inconsistency index had been used. The indices developed here made it possible to quantify and compare the FPV ranking disorders. In general, the method and metrics developed in this study went beyond the traditional focus of checking the Consistency Index (CI). A complete and detailed analysis of rank disturbances was feasible due to this new proposed methodology. This proposed methodology focused on an effective evaluation of the disturbances that occurred with the FPV and brings clarity to the final analysis of the outputs provided by the AHP and similar MCDA techniques. The analysis of the proposed NRI, IR and RQV indices gives a complete view of the ranking disturbance suffered by the FPV when compared to the original ranking. The analysis of these indices allows the DM to interpret whether a given rank reversal should be rejected or, on the contrary, reflects the expected response. A FPV with some rank reversion and a small NRI may represent the appropriate response and, if followed by a high NRI, should be rejected. The magnitude of the RQV also supports this analysis. Other surveys may carry out further investigations into the limits of acceptance of the FPV based on these proposed new indices. IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 14 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 REFERENCES Álvares, E., Giacometti, C., Gusso, E. (2008) Governança corporativa: um modelo brasileiro. 4th ed. Rio de Janeiro: Elsevier Editora. ISBN 978-85-352-3092-5, 8535230920 2008. Amenta, P., Lucadamo, A., Marcarelli, G. (2018). Approximate thresholds for Salo- Hamalainen Index. IFAC papersOnline, 51(11), 1655-1659. Doi: https://doi.org/10.1016/j.ifacol.2018.08.219. Araujo, M., Oliveira, E., Monteiro, S. (2017). Avaliação de maturidade de processos de gestão de riscos de TI: ferramenta de apoio para a qualidade e eficiência do processo. Revista Brasileira de Computação Aplicada, 9(2017). Doi: https://doi.org/10.5335/rbca.v9i2.6099. Atz, F., Gerhard, M., Freitag, V., Vanti, A., et al. (2017). Análise do relacionamento existente entre Governança da Tecnologia da Informação, o gerenciamento de risco corporativo e as funções de Compliance. In: Seminário de Administração. São Paulo: USP, 1-16. Belton, V., Gear, T. (1985). The legitimacy of rank reversal - a comment. International Journal of Management Science, 13, 143-144. Doi: https://doi.org/10.1016/0305-0483(85)90052-0 Bernasconi, M. Choirat, C., Seri, R. (2009). A re-examination of the algebraic properties of the AHP as ratio-scaling technique. Journal of Mathematical Psychology, 55(2), 152-165. Doi: https://doi.org/10.1016/j.jmp.2010.11.002 Bernasconi, M., Choirat, C., Seri, R. (2010). The Analytic Hierarchy Process and the theory of measurement. Management Science, 56(4), 699-711. Doi: https://www.jstor.org/stable/27784145. Bezerra, P. (2021). Proposição de um novo modelo para avaliar a sustentabilidade empresarial. 77 f. Dissertação (Mestrado em Administração) – Centro de Humanidades, Universidade Federal de Campina Grande, Campina Grande. Disponível em: http://dspace.sti.ufcg.edu.br:8080/jspui/handle/riufcg/17903. Acesso em: 20 jun. 2021. Botelho, M.(2021). A decisão multicritério nos conselhos de administração via análise hierárquica de processos (AHP): análise da inconsistência nas matrizes de comparação par a par nas abordagens de governança corporativa. 106 f. Dissertação (Mestrado em Engenharia e Gestão de Processos e Sistemas) - Instituto de Educação Tecnológica. Doi: https://doi.org/10.19146/pibic-2017-78788 Bozóki, S.; Rapcsák, T. (2008). On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices. Journal of Global Optimization, 42, 157-17. Doi:10.1007/s10898-007-9236-z. https://doi.org/10.1016/j.ifacol.2018.08.219 https://doi.org/10.5335/rbca.v9i2.6099 https://www.jstor.org/stable/27784145 http://dspace.sti.ufcg.edu.br:8080/jspui/handle/riufcg/17903.%20Acesso https://doi.org/10.19146/pibic-2017-78788 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 15 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Brunelli, M. (2018). A survey of inconsistency indexes for pairwise comparisons. International Journal of General Systems, 47(8), 751-771. Doi: https://doi.org/10.1080/03081079.2018.1523156. Brunelli, M. (2017). Studying a set of properties of inconsistency indexes for pairwise comparisons. Annals of Operations Research, 248, 143-161. Doi: https://doi.org/10.1007/s10479-016-2166-8. Buede, D.M., Maxwell, D.T. (1995). Rank disagreement: A comparison of Multi- Criteria methodologies. Journal of Multicriteria Decision Analysis, 4(1), 1-21. Doi: https://doi.org/10.1002/mcda.4020040102 Caríssimo, C., Moreira, M., Ornelas, M., Silva, J. (2016). Uso da Análise Hierárquica (AHP) para identificação da preferência de Peritos-Contadores quanto ao método de avaliação de sociedades em perícias contábeis. Revista de Educação e Pesquisa em Contabilidade, 10(1), 44-62. Doi: https://doi.org/10.17524/repec.v10i1.1333. Cobo, A., Vanti, A., Rocha, R. (2014). A fuzzy multicriteria approach for IT governance evaluation system. Journal of Information Systems and Technology Management, 11(2), 257-276. Doi: 10.4301/S1807-17752014000200003. Duarte, B. (2019). Maturidade da governança corporativa e gestão dos incentivos fiscais como determinantes da longevidade e desempenho das empresas do polo industrial de Manaus, Biblioteca FACE / UFG LVR 040/2020. Ellermeier, W., Faulhammer, G. (2000). Empirical evaluation of axioms fundamental to Stevens's ratio-scaling approach: I. Loudness production. Perception & Psychophysics, 62(8), 1505-1511, 2000. Doi: https://doi.org/10.3758/BF03212151. Emrouznejad, A., Marra, M. (2017). The state of the art development of AHP (1979 - 2017): a literature review with a social network analysis. International Journal of Production Research, 55(22), 6653-6675. Doi: https://doi.org/10.1080/00207543.2017.1334976. Franeka, J., Krestaa, A. (2014). Judgment scales and consistency measure in AHP. Procedia Economics and Finance, 12(2014), 164-173. Doi: 10.1016/S2212- 5671(14)00332-3. Gass, S., Rapcsá, K. (2001). Models, methods, concepts & applications of the Analytic Hierarchy Process: Singular value decomposition in AHP. International Symposium on the Analytic Hierarchy Process, 6, Berne, Switzerland, 129-130. Goepel, K. (2018). Comparison of judgement scales of the analytical hierarchy process: a new approach. International Journal of Information Technology and Decision Making, 18(2), 445-463. Doi:10.1142/S0219622019500044. Gonzáles, P., Vanti, A., Seiber, T. R., Castro, M. (2018). Design de hierarquização multicritério para o aumento da competitividade empresarial. International Conference on Information Systems & Technology Management, 5, 5025-5042. Doi:10.5748/9788599693148-15CONTECSI/DOCT-5997. https://doi.org/10.1080/03081079.2018.1523156 https://doi.org/10.1007/s10479-016-2166-8 https://doi.org/10.1002/mcda.4020040102 https://doi.org/10.17524/repec.v10i1.1333 https://doi.org/10.3758/BF03212151 https://doi.org/10.1080/00207543.2017.1334976 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 16 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Görener, A. (2012). Comparing AHP and ANP: An application of strategic decision making in a manufacturing company. International Journal of Business and Social Science, 3(11), 194-208. Doi: 10.1016/j.sbspro.2012.09.1139. Grybowski, A., Starczewski, T. (2020). New look at the inconsistency analysis in the pairwise-comparisons: based prioritization problems. Expert Systems with Applications, 159(2020). Doi: https://doi.org/10.1016/j.eswa.2020.113549. Grybowski, A., Starczewski, T. (2016). New results on inconsistency indexes and their relationship with the quality of priority vector estimation. Expert Systems Applications, 43, 197-212. Doi: 10.1016/j.eswa.2015.08.049. Harker, P., Vargas, L. (1987). The theory of ratio scale estimation: Saaty's Analytic Hierarchy Process. The Institute of Management Science, 33(11), 1383-1403. Doi:10.1287/mnsc.33.11.1383. IBGC (2015a). Código das melhores práticas de governança. 5th ed. São Paulo: Instituto Brasileiro de Governança Corporativa. ISBN: 978-85-99645-38-3 1. IBGC (2015b). Métrica de governança corporativa empresas de capital fechado. São Paulo: Instituto Brasileiro de Governança Corporativa. ISBN: 978-85-99645-38-3 1. Instituto Ethos (2013). Indicadores ETHOS para negócios sustentáveis e responsáveis. São Paulo: Instituto Ethos. Ishizaka, A., Labib, A. (2011). Review of the main developments in the Analytic Hierarchy Process. Expert Systems with Applications, 38(11), 14336-14345. Doi:10.1016/j.eswa.2011.04.143. Januri, S., Pozi, M., Nadzri, M., Zainuddin, M. (2021). Factors influencing Islamic banking by customers: A case study in Northern Malaysia. International Journal of Academic Research in Business e Social Sciences, 11(7), 720-735. Doi:10.6007/IJARBSS/v11-i7/10530 Kaiser, R., Futami, A., Valentina, L., Oliveira, M. (2019). Development of a managerial tool for prioritization and selection of portfolio projects using the Analytic Hierarchy Process methodology in software companies. Gestão & Produção, 26(4), 1-13. Doi: https://doi.org/10.1590/0104-530X4267-19. Kamil, K., Ismail, A., Shahimi, S. (2013). Objectives of Islamic banks in the management of asset and liability: a decision process of deriving priority. Proceedings of the International Symposium on the Analytic Hierarchy Process. Kazibudzki, P. (2017). New results of the quality of recently introduced index for a consistency control of pairwise judgments. Multi Criteria Decision Making, 12, 90- 102. Doi: 10.22367/mcdm.2017.12.07. Kazibudzki, P. (2017). The quality of priority ratios estimation in relation to a selected prioritization procedure and consistency measure for a Pairwise Comparison Matrix. Advances in Operations Research, 2019, 1-24. Doi:10.1155/2019/3574263. https://doi.org/10.1016/j.eswa.2020.113549 https://doi.org/10.1590/0104-530X4267-19 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 17 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Klozikova J., Dockalikova, I. (2014). Corporate governance rating: Synthesis of rating models of corporate governance with utilization methods AHP and DEMATEL. European Conference on Management, Leadership & Governance. Koklu, O., Jakubowski, E., Servi, T., Huang, J. (2016). Identifying and interpreting subjective weights for cognitive and performance characteristics of mathematical learning disability: An application of a relative measurement method. The Journal of International Lingual, Social and Educational Sciences, 2(2), 136-151. Lambert-Torres G., Costa C., Barros, M. & Martins,H. (2007). Processo de Análise Hierárquica Paraconsistente. VI Congress of Logic Applied to Technology, Paper 132. Doi: http://dx.doi.org/10.13140/RG.2.1.2022.6807 Loostsma, F. (1990). The French and the American school in multicriteria decision analysis. Rairo: Recherche Opérationnelle, 24(3), 263-285. Melo, R.T., Jesus, V. M. (2019). Avaliação 360º dos membros de um conselho de administração pelo método ANP. Revista de Ciência, tecnologia e Inovação, 26(4), 44-53. Doi: https://doi.org/10.1590/0104-530X4267-19. Neves, R., Pereira, V., Costa, H. (2015). Auxílio multicritério à decisão aplicado ao planejamento e gestão na indústria de petróleo e gás. Scientific Eletronic Library on Line -SCIELO, Associação Brasileira de Engenharia de Produção, 25(1). Doi: https://doi.org/10.1590/S0103-65132013005000060. Odic, D., Im, H., Eisinger, R., Ly, R., Halberda, J. (2015). PsiMLE: a maximum likelihood estimation approach to estimating psychophysical scaling and variability more reliably, efficiently, and flexibly. Behavior Research Methods, 48, 445-462. Doi: 10.3758/s13428-015-0600-5. Paiva, B. (2021). Mensuração do grau de maturidade da governança corporativa em cooperativas de crédito pelo método Análise Hierárquica de Processos – AHP. 2018. 154 f. Tese (Doutorado em Administração) – Pontifícia Universidade Católica de São Paulo, São Paulo, 2018. Disponível em: https://tede2.pucsp.br/handle/handle/19566. Acessed on June 20, 2021. Rodriguez, D. (2016). Autoavaliação da governança de TI em uma autarquia federal com base na gestão de riscos. 2016. 228 f. Dissertação (Mestrado em Computação Aplicada) - Instituto de Ciências Exatas, Departamento de Ciência da Computação, Universidade de Brasília, Brasília. Saaty, R. (2016). Decision making in complex environments. Pittsburgh, Pennsylvania, USA: Creative Decisions Foundation. ISBN 1-888603-00-3. Saaty, R.W. (1987). The Analytic Hierarchy Process: what is and how it is used. Mathematical Modeling, 9(3-5), 161-176. Doi:10.1016/0270-0255(87)90473-8. http://dx.doi.org/10.13140/RG.2.1.2022.6807 https://doi.org/10.1590/0104-530X4267-19 https://tede2.pucsp.br/handle/handle/19566 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 18 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Saaty, T.L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234-281. Doi: https://doi.org/10.1016/0022- 2496(77)90033-5. Saaty, T.L. Transport planning with multiple criteria: The Analytic Hierarchy Process applications and progress review. Journal of Advanced Transportation, 29(1), 81- 126. Doi: https://doi.org/10.1002/atr.5670290109. Saaty, T.L., Vargas, L.G. (2001). The seven pillars of the analytic hierarchy process. In: Models, methods, concepts & applications of the analytic hierarchy process. Boston, MA: Springer, 27-46. (International Series in Operations Research & Management Science; 34). Doi: https://doi.org/10.1007/978-1-4615-1665-1_2. Salo, A., Hämäläinen, P. (1997). On the measurement of preferences in the Analytic Hierarchy Process. Journal of Multi-Criteria Decision Analysis, 6(6), 309-319. Doi: https://doi.org/10.1002/(SICI)1099-1360(199711)6:6<309::AID- MCDA163>3.0.CO;2-2. Salo, A., Hämäläinen, P. (1993). Preference programming through approximate rate comparisons. European Journal of Operational Research, 82(3), 458-475. Santiago, L.P., Soares, V.O. (2018). Strategic alignment of an R&D portfolio by crafting the set of buckets. IEE Transactions of Engineering Management. Doi:10.1109/TEM.2018.2876408. Starczewski, T. (2016). Relationship between priority ratios disturbances and priority estimation errors. Journal of Applied Mathematics and Computational Mechanics, 15, 143-154. Doi: 10.17512/jamcm.2016.3.14. Starczewski, T. (2018). Remarks about geometric scale in the Analytic Hierarchy Process. Journal of Applied Mathematics and Computational Mechanics, 17(3), 71- 82. Doi: 10.17512/jamcm.2018.3.07. Subramanian, N., Ramanathan, R. (2012). A review of applications of Analytic Hierarchy Process in operations management. International Journal of Production Economics, 138(2), 215-241. Doi: http://dx.doi.org/10.1016/j.ijpe.2012.03.036 Trindade, J. (2016). Mensuração e avaliação da capacidade inovativa de micro, pequenas e médias empresas: aplicação de métodos multicritério fuzzy de apoio à decisão. 107 f. Dissertação (Mestrado em Qualidade e Inovação) – Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro. Doi: https://doi.org/10.17771/PUCRio.acad.29381. Tsyganok, V.V., Kadenko, S.V., Andriichuk, O.V. (2012). Significance of expert competence consideration in group decision making using AHP. International Journal of Production Research, 50(17), 4785-4792. Doi: https://doi.org/10.1080/00207543.2012.657967. https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1002/atr.5670290109 https://doi.org/10.1007/978-1-4615-1665-1_2 https://doi.org/10.1002/(SICI)1099-1360(199711)6:6%3c309::AID-MCDA163%3e3.0.CO;2-2 https://doi.org/10.1002/(SICI)1099-1360(199711)6:6%3c309::AID-MCDA163%3e3.0.CO;2-2 http://dx.doi.org/10.1016/j.ijpe.2012.03.036 https://doi.org/10.17771/PUCRio.acad.29381 https://doi.org/10.1080/00207543.2012.657967 IJAHP Article: Botelho/Analyzing the AHP priority vectors: going beyond inconsistency indexes International Journal of the Analytic Hierarchy Process 19 Vol. 14 Issue 2 2022 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 Vidal, L.A., Marle, F., Bocquet, J.C. (2011). Using a Delphi process and the Analytic Hierarchy Process (AHP) to evaluate the complexity of projects. Expert Systems with Applications, 38(5), 5388-5405. Doi: https://doi.org/10.1016/j.eswa.2010.10.016. Wang X., Triantaphyllou, E. (2008). Ranking irregularities when evaluating alternatives by using some ELECTRE methods. Omega – The International Journal of Management Science, 36(1), 46-63, 2008. Doi:10.1016/j.omega.2005.12.003 Watson, R., Freeling, N. (1982). Assessing attribute weights by ratios. Omega,20(6), 582-583 1982. Doi: http://dx.doi.org/10.1016/0305-0483(82)90061-5. Wegner, R., Battist, I.A., Tontini, J., Malheiros, M., Rossato, V. (2020). Aplicação do método Analytic Hierarchy Process (AHP) na priorização das ações de inovações em serviços em um estudo de multicaso. Navus - Revista de Gestão e Tecnologia, 10, 01- 19. Doi:10.22279/navus. 2020. v10.p01-19.1006. Yrjölä, M. Uncovering executive prioritization: Evaluating customer value propositions with pairwise comparison method. Journal of Service and Science Management, 8, 1-13. Doi: 10.4236/jssm.2015.81001. https://doi.org/10.1016/j.eswa.2010.10.016 http://dx.doi.org/10.1016/0305-0483(82)90061-5