INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL
ISSN 1841-9836, 9(4):408-418, August, 2014.
Simulation Experiments for Improving the Consistency Ratio of
Reciprocal Matrices
D. Ergu, G. Kou, Y. Peng, X. Yang
Daji Ergu
Southwest University for Nationalities
4th Section,Yihuan Nanlu,Chengdu, 610041, China, ergudaji@163.com
Gang Kou ∗
School of Business Administration
Southwestern University of Finance and Economics
No.555, Liutai Ave, Wenjiang Zone,Chengdu, 610054, China
*Corresponding author kougang@swufe.edu.cn
Yi Peng
School of Management and Economics
University of Electronic Science and Technology of China
No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, 611731, China
pengyicd@gmail.com
Xinfeng Yang
School of Statistics, Southwestern University of Finance and Economics
No.555, Liutai Ave, Wenjiang Zone, Chengdu, 610054, China, 315425159@qq.com
Abstract: The consistency issue is one of the hot research topics in the analytic
hierarchy process (AHP) and analytic network process (ANP). To identify the most
inconsistent elements for improving the consistency ratio of a reciprocal pairwise
comparison matrix (PCM), a bias matrix can be induced to efficiently identify the
most inconsistent elements, which is only based on the original PCM. The goal of this
paper is to conduct simulation experiments by randomly generating millions numbers
of reciprocal matrices with different orders in order to validate the effectiveness of
the induced bias matrix model. The experimental results show that the consistency
ratios of most of the random inconsistent matrices can be improved by the induced
bias matrix model, few random inconsistent matrices with high orders failed the
consistency adjustment.
Keywords: Reciprocal random matrix, Consistency ratio, induced bias matrix, simu-
lation experiment; analytic hierarchy process (AHP)/analytic network process (ANP)
1 Introduction
In the analytic hierarchy process (AHP) and analytic network process (ANP), the pairwise
comparison matrix (PCM hereinafter) originated by Thurstone [1] is one of the most important
components, which is used to compare two criteria or alternatives with respect to a given criterion
, then a matrix A = (aij)n×n is built to reflect the direct and indirect judgment relations between
pairs of criteria or alternatives with respect to a given criteria, where aij > 0, aij = 1aji . All PCMs
are then used to derive the priority vectors, and the alternatives can be ranked by aggregating
the local priority vectors [2–4]. However, the decision made based on the final priority vectors
is effective only if the paired comparison matrices pass the consistency test [5]. In practice,
it is usually difficult to obtain a matrix that satisfies the perfect consistency condition (i.e.
aij = aikakj for i, j, k = 1, 2, . . . , n). Therefore, Saaty [6, 7] proved that the maximum eigenvalue
λmax of matrix A always satisfies λmax ≥ n and the equality holds if and only if A is perfectly
Copyright © 2006-2014 by CCC Publications
Simulation Experiments for Improving the Consistency Ratio of Reciprocal Matrices 409
consistent. Based on this property, Saaty proposed the consistency ratio (CR) to measure the
consistency of a matrix, i.e. the consistency of a matrix is acceptable if the CR is less than
0.1. However, this condition sometimes cannot be satisfied with because of the limitations of
experiences and expertise, prejudice as well as the complexity nature of the decision problem [8].
To improve the consistency ratio of a matrix, many models and methods have been proposed
over the past few decades. For instance, Harker [9] regarded the largest absolute value(s) in
matrix
{
viωj − a2jivjωi
}
for all i, where i > j , as the most inconsistent element(s). Saaty
[6] constructed the deviation differences matrix B = [bij] = [|aij − ωi/ωj|] to identify the
most inconsistent entry, where ωi and ωj are any two subjective priority weights in the ω =
(ω1, · · · , ωn) . Based on these models, Xu and Wei [10] generated a near consistent matrix
B = (aλij(ωi/ωj)
1−λ)n×n to improve the consistency, where λ is a parameter of the auto-adaptive
algorithm. Besides, Saaty [7] and Cao et al. [11] introduced Hadamard operator “ ◦ ” to
build a perturbation matrix E and a deviation matrix A, in which E = (aij) ◦ (ωj/ωi)] and
A = [ωi/ωj] ◦ [aij/(ωi/ωj)] , to identify the most inconsistent entry.
There is a common feature in the previously reviewed models, that is, these models are
dependent on the priority weights ωi and ωj, but there exist more than 20 priority derivation
methods [12–14], and the final priority weights obtained from different methods might be different
when the matrix is inconsistent. Therefore, Ergu et al. [8] proposed an induced bias matrix to
identify the most inconsistent entry in the original inconsistent matrix A. To do so, three major
steps containing seven specific steps were proposed and several numerical examples were used to
validate the proposed model. In this paper, we attempt to conduct simulation experiments to
further validate the effectiveness of the proposed induced bias matrix (IBM) model by generating
randomly millions number of the reciprocal positive matrices with different orders. The step 6
and step 7 proposed in Ergu et al. [8] are further quantified and detailed in order to implement
automatically modification.
The remaining parts of this paper are organized as follows. The next section briefly describes
the induced bias matrix model. The simulation experiments and some algorithms are performed
and proposed in Section 3. Section 4 concludes the paper as well as future research directions.
2 The induced bias matrix model
In Ergu et al. [8], the theorem of induced bias matrix and two corollaries were proposed
to identify the most inconsistent entries in a PCM and improve the consistency ratio. For the
readers’ convenience, we first briefly describe the related theorem and corollaries of the IBM
model as preliminary of IBM model.
The Theorem 1 says that "the induced matrix C=AA-nA should be a zero matrix if comparison
matrix is perfectly consistent". Based on this theorem, if comparison matrix A is approximately
consistent, Corollary 1 derived that "the induced matrix C=AA-nA should be as close as possible
to zero matrix". However, if the pairwise matrix is inconsistent, Corollary 2 says that "there
must be some inconsistent elements in induced matrix C deviating far away from zero". By this
corollary, the largest value in matrix C can be used to identify the most inconsistent element in
the original matrix A. Some of the identification processes are presented next.
The procedures of the IBM model proposed in Ergu et al. [8] include three major steps,
containing seven specific steps (Details are referred to Ergu et al. [8]). The first five steps are
easy to be implemented by MATLAB software in practice, i.e. 1) Construct an induced matrix
C=AA-nA; 2) Identify the largest absolute value(s) of elements and record the corresponding
row and column; 3) Construct the row vector and column vector using the recorded location; 4)
Calculate the scalar product f of the vectors; 5) Compute the deviation elements between aij
410 D. Ergu, G. Kou, Y. Peng, X. Yang
and vectors f. However, for Steps 6-7, the definitions are not easy to be quantified and it needs
the decision makers to determine when we should use Method of Maximum,Method of Minimum,
and Method for adjusting aij to identify the most inconsistent entries. In the following section,
we combine these identification methods to perform the simulation experiment by generating
randomly reciprocal matrix in order to validate the effectiveness of the induced bias matrix
model.
3 Simulation experiments
3.1 Design of simulation experiments
The simulation experiments were performed to confirm the effectiveness of the induced bias
matrix model using random inconsistent reciprocal matrices. We generated randomly 106 set
of reciprocal matrices with orders 3 to 9, and 105 set of reciprocal matrices with orders 10-12,
i.e. the entries above the main diagonal of a reciprocal matrix is generated randomly from
the 17 numbers (1/9, 1/8, 1/7, . . . , 1, 2, 3, . . . , 9) in order to satisfy the Saaty’s fundamental 9-
point scales, the entries below the main diagonal of the PCM is filled automatically with the
corresponding reciprocal value. Then calculating the consistency ratio by the formula proposed
by Saaty [7], where λmax is the maximum eigenvalue of matrix A, and n is the order of matrix
A. If the CR < 0.1, discard the generated matrix, if the CR ≥ 0.1, then applying the IBM
model to modify the inconsistent entry and improve the consistency ratio by the six steps and
the combined algorithm, as shown in Figure 1. If the consistency ratio of the generated randomly
reciprocal pairwise comparison matrix cannot be reduced to be lower than 0.1, then record the
corresponding matrix. The specific procedures of this simulation experiment are shown in Figure
1.
Figure 1: Procedures of simulation experiment by IBM model
For the matrices with CR ≥ 0.1, the formula of steps 1-6 presented in Ergu et al. [8] are
directly used to identify and modify the most inconsistent entries in matrix A as well as improving
the consistency ratio. However, the Method of Maximum, Method of Minimum and Method for
Simulation Experiments for Improving the Consistency Ratio of Reciprocal Matrices 411
identifying aij proposed in Step 6 and Step 7 involve qualitative observation and judgment, for
instance, how many absolute values in vector f can be regarded as more absolute values that are
around zero? How to measure the absolute values of aij, aik and akj are too large or too small
by quantifying terms? Therefore, the following Algorithm is used to combine the previously
mentioned identification processes.
3.2 Algorithms of simulation experiments
In order to simulate the induced bias matrix model, the program codes with two input
parameters, n and n1 were written by Matlab 7.0 to randomly generate reciprocal matrices, in
which n denotes the numbers of random reciprocal matrix, while n1 represents the number of
simulation. For the space limitations, we omitted the first five steps, and the following Algorithm
is used to combine the Method of Maximum,Method of Minimum, and Method for adjusting aij.
Algorithm 1: Improving the consistency ratios of the random reciprocal matrices with
CR ≥ 0.1
Input: Matrix Order, n; Number of simulation, n1
Output: Matrices with CR ≥ 0.1
Process:
01. C=AA-nA % Matrix A is the generated randomly reciprocal matrix with CR ≥ 0.1
02. If cij < 0
03. Adjust aij using aij = 1n−2
∑n
k=1 aikakj
04. End % Method for identifying aij(1)
05. If cij > 0 & & min(f) == 0 % We can obtain that aij is inconsistent whether it is too
large or too small, in which f is the vector product
06. Adjust aij using ãij = 1n−2
∑n
k=1 aikakj
07. End % Method for identifying aij(2)
08. If cij > 0 % aij and aik (or akj ) might have problematic
09. [m, k] = max(f); % m is the element with the largest value in vector f, while k
is the corresponding location.
10. If cik < 0 & & ckj ≥ 0 % aik is problematic (too large).
11. Adjust cik using aik + cik/(n − 2).
12. Break
13. End
14. If cik ≥ 0& & ckj < 0 % akj is problematic and large
15. Adjust akj using akj + ckj/(n − 2)
16. Break
17. End
19. If cik < 0 & & ckj < 0 % cik and ckj are problematic
20. If abs(cik) >= abs(ckj)
21. Adjust aik using aik + cik/(n − 2)
22. Break
23. Else
24. Adjust akj using akj + ckj/(n − 2)
25. Break
26. End
27. End
412 D. Ergu, G. Kou, Y. Peng, X. Yang
28. If f(k) > 0 & & cik >= 0 & & ckj >= 0 % It is unreasonable to occur simultaneously,
if it does occur, then go to adjust the second largest value.
29. cij = 0;
30. End
31. End
32. Calculate the CR; % see Algorithm 2
33. If CR < 0.1
34. Break
35. End.
Algorithm 2: Calculating the consistency ratio of the modified matrix B
Input: Modified random matrix B
Output: Consistency ratio CR.
Process:
01. n=length(B); % B is reciprocal matrix
02. RI=[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 0 0.52 0.89 1.12 1.26 1.36 1.41 1.46 1.49 1.52 1.54
1.56 1.58 1.59];
03. [a,b]=eig(B);
04. [c1,d1]=max(b); % c1 is maximum value in each column, d1 is the corresponding row
of each element
05. [e1,f1]=max(c1); % e1 is the largest element, f1 is the corresponding column.
06. CI=(e1-n)/(n-1);
07. CR=CI/RI(2,n);
3.3 Experimental results
In this section, we do not attempt to optimize the program codes for speed, therefore, we set
the matrix order n to be 3-12, and the simulation number n1=106 for the matrices with orders
3-9. For the matrix with orders 9-12, we only simulated 105 numbers of randomly reciprocal
matrices. The results of simulation experiments are shown in Table 1. It can be seen that
some of the random reciprocal matrices with orders from 3 to 6 passed the consistency test, for
instance, 206130 random matrices with order 3 passed the consistency test among 106 matrices,
while 73 random matrices with order 6 passed the consistency text. However, all random matrices
with orders 7-12 did not pass the consistency test. For the random matrices with CR ≥ 0.1, the
proposed IBM model was used to modify the most consistent entries and improve the consistency
ratio. Table 1 shows that the consistency ratios of all the inconsistent random matrices with
CR ≥ 0.1 and orders 3-7 have been improved and lower than 0.1 after the proposed IBM model
is used to modify the random matrices, as shown in Figures 2-7, while some matrices still failed
the consistency test, the numbers are 3 for order 8, 5 for order 9, 1 for order 10, 2 for order 11
and 13 for order 12, as shown in Figures 8-12. The corresponding simulation Figures for 106
random matrices with orders 3 to 9, and 105 random matrices with orders 10-12 are shown in
Figures 2-10.
Simulation Experiments for Improving the Consistency Ratio of Reciprocal Matrices 413
Table 1 Simulation experiments for randomly generated matrices with different orders
Matrix Order Number of simula-
tion
Number of matri-
ces with CR ≥ 0.1
Failed matrices Succeeded Matrices
3 1000000 793870 0 793870
4 1000000 968083 0 968083
5 1000000 997518 0 997518
6 1000000 999927 0 999927
7 1000000 1000000 0 100000
8 1000000 1000000 3 999997
9 1000000 1000000 5 999995
10 100000 100000 1 99999
11 100000 100000 2 99998
12 100000 100000 13 99987
0 2 4 6 8 10
x 10
5
0
0.5
1
1.5
2
2.5
CR>0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
0 2 4 6 8 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 2: Simulation experiment for 106 randomly generated matrices with order 3
0 5 10
x 10
5
0
0.5
1
1.5
2
2.5
CR>0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
0 5 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 3: Simulation experiment for 106 randomly generated matrices with order 4
414 D. Ergu, G. Kou, Y. Peng, X. Yang
0 5 10
x 10
5
0
0.5
1
1.5
2
2.5
CR>0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
0 5 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR after adjustment
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 4: Simulation experiment for 106 randomly generated matrices with order 5
0 5 10
x 10
5
0
0.5
1
1.5
2
2.5
CR>0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
0 5 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR after adjustment
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 5: Simulation experiment for 106 randomly generated matrices with order 6
0 5 10
x 10
5
0
0.5
1
1.5
2
2.5
CR>0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
0 5 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR after adjustment
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 6: Simulation experiment for 106 randomly generated matrices with order 7
Simulation Experiments for Improving the Consistency Ratio of Reciprocal Matrices 415
3 4 5 6 7 8 9 10
x 10
5
X: 9.816e+005
Y: 0.1768
CR>0.1
The number of simulation
X: 3.154e+005
Y: 0.1743
X: 2.589e+005
Y: 1.85 X: 6.034e+005
Y: 1.811
0 1 2 3 4 5 6 7 8 9 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
X: 6.259e+005
Y: 0.0109
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
X: 4.691e+005
Y: 0.01213
X: 4.669e+004
Y: 0.01128
Figure 7: Simulation experiment for 106 randomly generated matrices with order 8
3 4 5 6 7 8 9 10
x 10
5
CR>0.1
The number of simulation
X: 9.508e+005
Y: 0.2731
X: 2.771e+005
Y: 0.2575
X: 9.83e+005
Y: 1.685
X: 4.352e+005
Y: 1.7
X: 1.862e+005
Y: 1.708
0 1 2 3 4 5 6 7 8 9 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 8: Simulation experiment for 106 randomly generated matrices with order 9
2 3 4 5 6 7 8 9 10
x 10
5
CR>0.1
The number of simulation
0 1 2 3 4 5 6 7 8 9 10
x 10
5
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 9: Simulation experiment for 105 randomly generated matrices with order 10
416 D. Ergu, G. Kou, Y. Peng, X. Yang
3 4 5 6 7 8 9 10
x 10
4
X: 2.42e+004
Y: 1.436
CR>0.1
The number of simulation
X: 5.303e+004
Y: 1.467
X: 8.631e+004
Y: 1.48
X: 1.849e+004
Y: 0.5031 X: 9.352e+004
Y: 0.4573
X: 2.631e+004
Y: 0.4731
0 1 2 3 4 5 6 7 8 9 10
x 10
4
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
X: 1.612e+004
Y: 0.03592
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 10: Simulation experiment for 105 randomly generated matrices with order 11
2 3 4 5 6 7 8 9 10
x 10
4
CR>0.1
The number of simulation
0 1 2 3 4 5 6 7 8 9 10
x 10
4
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
CR<0.1
T
h
e
v
a
lu
e
o
f
C
R
The number of simulation
Figure 11: Simulation experiment for 105 randomly generated matrices with order 12
Simulation Experiments for Improving the Consistency Ratio of Reciprocal Matrices 417
4 Conclusions
In this paper, some of the identification processes proposed in Ergu et al. [8] were combined
to implement the programming. Based on these combinations, an algorithm was proposed and
simulation experiments on random reciprocal matrices with different orders were conducted to
validate the effectiveness of the induced bias matrix model. We found that some matrices gen-
erated randomly could pass the consistency test, and the higher the orders of matrices are, the
less the matrices have CR < 0.1. When the orders of random matrices increase to 7, all matrices
generated randomly have CR ≥ 0.1, and they need to be adjusted. After the algorithm of the
induced bias matrix (IBM) model was applied to these matrices, all the consistency ratios of
random matrices with orders 3-7 were improved and less than the threshold 0.1, while fewer
matrices with order higher than 8 still could not be modified satisfactorily. However, we believe
that the consistency of the pairwise comparison matrices provided by experts will be much better
than the consistency of random matrices, thus the proposed IBM model is capable of dealing
with the consistency of a PCM.
Although the results of the simulation experiments show the effectiveness of the IBM model,
the experimental findings also reveal the failed tendency will increase with the increase of the
matrices order. The failed matrices should be paid more attention to and analyze the reason
why it failed the consistency test, we leave it for further research in future.
Acknowledgements
This research was supported in part by grants from the National Natural Science Foundation
of China (#71222108 for G. Kou, #71325001 and #71173028 for Y. Peng, #71373216 for D.
Ergu, #91224001 for S.M.Li);Program for New Century Excellent Talents in University #NCET-
12-0086 for Y. Peng, and the Research Fund for the Doctoral Program of Higher Education
(#20120185110031 for G. Kou). We thank Prof. Waldemar W. Koczkodaj for encouraging us to
conduct simulation experiments in order to further validate the effectiveness of IBM model.
Bibliography
[1] Thurstone, L. (1927), A law of comparative judgment, Psychological review, 34(4): 273-273.
[2] Saaty, T. L. 1986. Axiomatic Foundation of the Analytic Hierarchy Process, Management
Science 32(7): 841-855.
[3] Saaty, T. L. (1994), How to Make a Decision: The Analytic Hierarchy Process, Interfaces,
24: 19-43.
[4] Siraj, S.; Mikhailov, L.; Keane, J. (2012), A heuristic method to rectify intransitive judgments
in pairwise comparison matrices, European Journal of Operational Research, 216: 420-428.
[5] Kou, G., Ergu,D., Shang,J. (2014). Enhancing Data Consistency in Decision Matrix: Adapt-
ing Hadamard Model to Mitigate Judgment Contradiction, European Journal of Operational
Research, 236(1):261-271.
[6] Saaty, T. L. (1980), The Analytical Hierarchy Process, New York: McGraw-Hill.
[7] Saaty, T. L. (2003). Decision-making with the AHP: Why is the principal eigenvector neces-
sary, European Journal of Operational Research, 145(1): 85-91.
418 D. Ergu, G. Kou, Y. Peng, X. Yang
[8] Ergu, D. ; Kou, G.; Peng, Y.; Shi, Y. (2011). A Simple Method to Improve the Consistency
Ratio of the Pair-wise Comparison Matrix in ANP, European Journal of Operational Research,
213(1): 246-259.
[9] Harker, P. T. 1987. Derivatives of the Perron root of a positive reciprocal matrix: With
applications to the analytic hierarchy process, Applied Mathematics and Computation, 22:
217-232.
[10] Xu, Z., Wei, C. (1999),A consistency improving method in the analytic hierarchy process,
European Journal of Operational Research, 116: 443-449.
[11] Cao, D., Leung, L. C., Law, J. S. (2008). Modifying inconsistent comparison matrix in
analytic hierarchy process: A heuristic approach, Decision Support Systems, 44: 944-953.
[12] Choo, E., Wedley, W. (2004). A common framework for deriving preference values from
pairwise comparison matrices, Computer and Operations Research, 31 (6): 893-908.
[13] Lin, C. (2007). A revised framework for deriving preference values from pairwise comparison
matrices, European Journal of Operational Research, 176 (2): 1145-1150.
[14] Kou,G, Lin,C. (2014). A cosine maximization method for the priority vector derivation in
AHP, European Journal of Operational Research, 235:225-232.