IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 469 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 Implementation of an Online Software Tool for the Analytic Hierarchy Process (AHP-OS) Klaus Goepel BPMSG (Business Performance Management Singapore) drklaus@bpmsg.com ABSTRACT The Analytic Hierarchy Process (AHP) remains a popular multi-criteria decision method. The author has implemented a free, web-based AHP online system with noteworthy features, allowing for the detailed analysis of decision problems. Besides standard functions, like flexible decision hierarchies, support to improve inconsistent judgments, and alternative evaluation and sensitivity analysis, the software can handle group input, calculate group consensus based on Shannon α and β-entropy and estimate weight uncertainties based on randomized small variations of input judgments. In addition, different AHP judgment scales can be applied a posteriori and alternative evaluation can be done using the weighted sum (WSM) or weighted product model (WPM). This flexibility opens up opportunities to study decision projects under various parameters. The author’s intention was to provide a complete and free software tool for educational and research purposes where calculations and algorithms are well documented and all input data and results can be exported in an open format for further processing or presentation. The article describes the basic concept and structure of the software and the underlying mathematical algorithms and methods. Challenges and practical experiences during the implementation, validation and productive phase of the software are highlighted. Keywords: Multi-criteria decision-making; Analytic Hierarchy Process; AHP software; AHP online system; AHP group decision making 1. Introduction The Analytic Hierarchy Process (AHP) was developed by T. L. Saaty in 1980 and remains a widespread multi-criteria decision method (MCDM) (Saaty, 1980). Based on pairwise comparison inputs, weights are calculated by finding the dominant right eigenvector (EV) of a positive reciprocal decision matrix. Calculations for simple decision problems can be done with a spreadsheet program, but for more complex decision problems a software tool is needed (Goepel, 2013). There are several AHP software packages available on the market (see Annex), and some of them have been described or compared in the literature (Ossadnik & Kaspar 2013; Ishizaka & Labib 2009; Siraj et al. 2015). Most of these packages target companies and pursue a business objective. Their focus lies on the application, and less on the methods and algorithms, which are often not fully transparent to users. The author’s intention was to provide a complete and free software tool for educational and research purposes where IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 470 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 methods and algorithms are well documented and validated. As a result, a web-based AHP online system (AHP-OS) was developed and is available in its full functionality to any user for non-commercial purposes. The software is implemented using PHP, an open source general-purpose, object oriented scripting language that is especially suited to web development. Database function calls use Structured Query Language (SQL), the most widely used database language with available open source database engines like SQLite or MariaDB. The whole package is developed from scratch, using only a few other available open source packages for general supporting functions, like sending e-mails or user login and registration (Annex 1). Functions and features of AHP-OS include the following:  Flexible definition of decision hierarchies as text input, following a simple syntax with multi-language support using Unicode character coding,  Weight calculation (hierarchy mode) and alternative evaluation (alternative mode) using the AHP eigenvector method,  Pairwise comparison input, highlighting the top-3 most inconsistent judgments,  Partial judgments,  A posteriori application of different AHP judgment scales,  Group decision making using weighted geometric mean aggregation of individual judgments (WGM-AIJ),  Group consensus calculation based on Shannon α and β-entropy,  Weight uncertainty estimation using Monte Carlo simulation,  Sensitivity analysis,  Weighted sum model (WSM) and weighted product model (WPM) for the aggregation of alternatives,  Export of input and result data as comma separated value (CSV) files for further processing or presentation in a spreadsheet program. The software has a modular structure and is implemented using object oriented programming (OOP). It consists of five major objects (classes): 1. ahp class for all basic AHP calculations, 2. ahpHierarchy class for decision hierarchy related functions, 3. ahpDb class for database handling, 4. ahpGroup class for result calculations and 5. ahpAdmin class for all administrative functions like user and project administration. These classes contain all AHP methods. In the following, we will describe the major functions in more detail. 2. Decision hierarchies and eigenvector calculations 2.1 Hierarchies Decision hierarchies in the program are defined using plain text input consisting of a sequence of statements following a simple syntax. This provides different advantages: IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 471 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590  Hierarchy definitions in plain text form can be modified or duplicated by simple copy and paste functions.  They can easily be archived as text files.  Unicode character coding allows for multi-language support; languages like Chinese, Korean, Thai etc. are supported.  Local weights can be predefined within the hierarchy text. This is useful when weights are already known, e.g. from a previous project or when they are derived from other MCDM methods, and AHP-OS is only used for alternative evaluation. The syntax is defined as follows: → ; [{;}] → : , [,] → { [ = ]} For all in a the sum of has to be one. A of the second and any further has to be one of the in . Each represents a decision matrix, and the corresponding priority vector’s dimension is the number of . Figure 1 shows a simple example of a two level hierarchy. In the first level (main criteria), weights for the two criteria are predefined as 30% and 70%. For the sub-criteria, weights are automatically set to the default value 1/nleaf. Figure 1 Example of a decision hierarchy with 2 levels, 3 nodes and 4 leafs This hierarchy in Figure 1 is defined by the following hierarchy text: AHP-project: Criterion-1=0.3, Criterion-2=0.7; Criterion-1: Sub-criterion A, Sub-criterion B; Criterion-2: Sub-criterion C, Sub-criterion D; All methods related to the decision hierarchy are coded in the ahpHierarchy class of the software. The class includes a parser to translate the hierarchy text into a multi- dimensional array. The parser checks for syntax errors and cleans the input text from redundant and impermissible characters. Supporting functions to extract nodes, branches or leafs from the hierarchy are included in this class. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 472 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 2.2 AHP eigenvector method Once a hierarchy is defined and stored in the database, weights can be calculated through the following steps. Each step is coded as a method in the ahp class of AHP-OS: 1. Get pairwise comparisons (pwc): function get_pair_comp() 2. Apply selected AHP judgment scale: function calcScale() 3. Fill decision matrix from pairwise comparisons: function getMatrixFromPwc() 4. Find the eigenvector using the power method: function set_evm_evec() 5. Calculate the dominant eigenvalue from the eigenvector: function setEvmEval() 6. Calculate the consistency ratio CR: function setAlonsoCr() 7. Calculate the inconsistency matrix: function get_incon_matrix() 8. Identify and highlight the top three inconsistencies: function get_inconsistency() 9. Go back to step 1 until the user submits his judgments. Figure 2 shows an example of the display of pairwise comparisons to the user. Figure 2 Pairwise comparison input All pairwise comparisons are internally stored in the format 𝑝𝑤𝑐 = (𝑎1,𝑎2, … 𝑎𝑛𝑝𝑐 ), (𝑥1,𝑥2, … 𝑥𝑛𝑝𝑐 ) (1) With integers 𝑎𝑖 ∈ [ 0,1], 𝑥𝑖 ∈ [1, 𝑀], M = 9 and 𝑖 = 1 … 𝑛𝑝𝑐 , with 𝑛𝑝𝑐 is the number of pairwise comparisons. 𝑛𝑝𝑐 = 𝑛2−𝑛 2 . (2) For n criteria the n x n decision matrix is then filled from pwc. For 𝑎𝑖 = 0 we take 𝑥𝑖 , for 𝑎𝑖 = 1 we have to take the reciprocal of 𝑥𝑖 . For example, for three criteria with 𝑝𝑤𝑐 = (0,0,1), (3, 5, 7) the decision matrix is M = ( 1 3 5 1 3 1 1 7 1 5 7 1 ) (3) The selected format minimizes redundancy and uses less memory than storing the complete positive reciprocal matrix. The dominant eigenvalue λ of M is calculated using the power method (Larsen, 2013). The number of iterations is limited to 20, this is sufficient for an accepted approximation error of 1.E-7. As a result we get the local priority vector with weights wi for criterion i = 1 to n. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 473 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 2.3 Consistency ratio Instead of Saaty’s original consistency ratio calculation (Saaty, 2008), based on the average random consistency index RIn 𝐶𝑅 = λ−𝑛 (𝑛−1)𝑅𝐼𝑛 (4) we use the linear fit proposed by Alonso and Lamata (2006) to calculate the consistency ratio CR 𝐶𝑅 = 𝜆−𝑛 2.7699 ∙ 𝑛−4.3513−𝑛 (5) It can be used for matrices larger than 10 x 10. In the case of a decision hierarchy with more than one node, CR of each hierarchy node is calculated, and for the global weights the program shows the maximum of all CRs. 2.4 Calculation of the top-3 inconsistencies If the consistency ratio CR exceeds 10%, the software calculates the top-3 inconsistent judgments. For all elements of the decision matrix dij we construct the inconsistency matrix based on the calculated weights wi and wj as 𝑒𝑖𝑗 = 𝑑𝑖𝑗 ∙ 𝑤𝑗 𝑤𝑖 (6) and identify the three largest 𝑒𝑖𝑗 . The corresponding pairwise comparisons are then highlighted for the decision makers, allowing them to make some adjustments (Saaty, 2003). IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 474 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 Table 1 AHP judgment scales implemented in the software No Name Scale function Comment 0 AHP scale 𝑐 = x Saaty (1980) 1 Logarithmic scale 𝑐 = log𝑎 (𝑥 + a − 1) Ishizaka et. al (2010) a = 2 2 Root square scale 𝑐 = √x 𝑎 Harker, Vargas (1987) we use a = 2 3 Inverse linear scale 𝑐 = 9 10 − 𝑥 Ma-Zheng (1991) 4 Balanced scale 𝑐 = 9 + 𝑥 11 − 𝑥 Salo, Hämäläinen (1997) for [0.1,0.9] 5 Generalized balanced scale 𝑐 = 9 + (𝑛 − 1)𝑥 9 + 𝑛 − 𝑥 n number of criteria Goepel (2019) 6 Adaptive-balanced scale 𝑐 = (9𝑛 − 10)(𝑥 − 1) + 80 (9𝑛 − 10)𝑥 − 89𝑛 + 90 (𝑛 − 1) Goepel (2019) 7 Adaptive scale 𝑐 = 𝑥 1+ ln (𝑛−1) ln 9 Goepel (2019) 8 Power scale 𝑐 = x𝑎 Harker, Vargas (1987) a = 2 9 Geometric scale 𝑐 = 𝑎𝑥−1 Lootsma (1994), a = 2 2.5 AHP judgment scales Over the last several decades, a variety of judgment scales, different from Saaty’s fundamental 1 to 9 scale, have been proposed. In AHP-OS, pairwise comparisons are stored with their original judgment values (Equation 1); therefore, we can apply different scales a posteriori and study the effect of different scales (Table 1) on the resulting weights. x is the judgment value (x = 1 … 9); c resp. 1/c the ratio used as entry into the decision matrix. Goepel (2019) gives a comparison of these scales, based on weight boundaries and weight ratio, weight uncertainty and weight dispersion. 3. AHP-OS database and data structure The software uses the PHP data object (PDO) interface to SQL databases. The current implementation can handle either two SQL servers, open source MariaDB, or the public domain SQLite database engine. The implemented database structure is comprised of four database tables: 1. Users: table for user registration and login. Registered users can be project authors and can store and manage their own AHP projects. 2. Projects: table of AHP projects with unique session code, project name, project description and hierarchy definition. Its foreign key is the user name from the table Users. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 475 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 3. Judgments (pwc): table containing all pairwise comparisons with the decision maker’s name and nodes of the decision hierarchy. Foreign key is the session code from the table Projects. 4. Alternatives: table with alternative names. Foreign key is the session code from the table Projects. This structure minimizes redundancies and keeps the database slim. In order to identify projects, each project gets a unique 6-character session code. As described before, decision hierarchies are defined with plain text input, and pairwise comparisons are stored in a simple integer format. For group decision making, participant’s judgments are stored in the pwc table. Participants do not need to register as users; they just need to follow a link with the unique session code provided by the AHP project author. They then can start to input their judgments by opening the link and providing their name. (Field participant in table pwc). The ahpDb software class handles the interfacing with the SQL database server. The following SQL statement is an example and will result in a table of all projects for a specified user name, showing the unique project session code, the project name, number of alternatives (if any), number of participants (decision makers) with pairwise comparison inputs, a short project description and the creation date/time of the project. The database does not store results; all results are calculated on the fly using methods in the software class ahpGroup. SELECT session_code, project_name, count(alt) AS Alt, count(pwpart) AS Part, description, date(date_time) FROM projects LEFT JOIN (SELECT DISTINCT pwc.session_code AS pwsc, pwc.participant AS pwpart FROM pwc) ON projects.session_code = pwsc LEFT JOIN (SELECT alternatives.session_code AS altsc, projects.session_code AS alt FROM projects, alternatives WHERE projects.session_code = alternatives.session_code GROUP BY alternatives.session_code ) ON projects.session_code = altsc WHERE projects.user_name = :user_name GROUP BY session_code ORDER BY projects.date_time DESC; 3.1 Basic data structure Each node in the decision hierarchy represents a decision matrix with the number of branches as number of criteria. For the example given in Figure 1, AHP project, criterion-1 and criterion-2 are the nodes of the hierarchy with two branches each, and sub-criterion A to D build the leafs of the decision hierarchy. In the software, we need to calculate the weights of each branch or leaf of all nodes and for all decision makers (participants). The weights (local priorities) are stored in a 3-dimensional array with array keys [participant], [node] and [leaf]. The array key [participant = 0] holds the consolidated group results; the array key [node = “pTot”] holds the global priorities. This data structure applies to both, hierarchy and alternative evaluation, where for the latter IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 476 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 the hierarchy’s leafs become nodes with alternatives as branches. For any calculation, the software has to loop through all nodes. Once the weight calculations are done, the results (local priorities) are inserted into the hierarchy definition text, for example Criterion-2 becomes Criterion-2=0.7. The modified hierarchy text can then be used to define alternatives and be stored as a new project in the database for alternative evaluation. 3.2 Partial judgments As pairwise comparisons for each node are stored in the pairwise comparisons database table, it is possible for participants to do a partial evaluation, i.e. to give their input only for a part of the hierarchy or the alternatives. This can be useful, if a project requires inputs from different experts with specific expertise in selected nodes of the decision hierarchy. 4. Group decision making 4.1 Aggregation of individual judgments One of the major advantages of a web based AHP software is that it allows global access. Once a decision hierarchy is defined and stored as a project on the server, decision makers can give their inputs from any location, as long as they have internet access. In AHP-OS, each project is identified by a unique session code and participants only need a link with this session code to start their pairwise comparisons and submit their judgments. The number of participants is practically unlimited. In order to consolidate all participant’s judgments, several group AHP aggregation techniques are available, but not all group AHP methods are equally convenient, and the selection of the method might depend on the specific application (Grošelj. P et al., 2015; Wen-Hsiang Wu et al., 2008). In the current version of the software, we use the weighted geometric mean aggregation of individual judgments (WGM-AIJ) as it is the only method that meets several required axiomatic conditions such as the reciprocal property. We calculate the geometric mean and standard deviation of all K participant’s individual judgments 𝑝𝑤𝑐𝑘 to form the consolidated decision matrix 𝑎𝑖𝑗 cons. Sum over K participants 𝑝𝑤𝑐𝑥 = ∑ ln(𝑝𝑤𝑐𝑘 ) 𝐾 𝑘=1 (7) Square sum over K participants 𝑝𝑤𝑐𝑥2 = ∑ [ln(𝑝𝑤𝑐𝑘 )] 2𝐾 𝑘=1 (8) Geometric mean 𝑝𝑤𝑐CONS = exp ( 𝑝𝑤𝑐𝑥 𝐾 ) (9) Standard deviation 𝑝𝑤𝑐SD = exp ( √ 𝑝𝑤𝑐𝑥2 − 1 𝐾 𝑝𝑤𝑐𝑥∙𝑝𝑤𝑐𝑥 𝐾−1 ) (10) Using the pairwise comparison data from Equation 9 to fill the consolidated decision matrix we get 𝑎𝑖𝑗 cons = (∏ 𝑎𝑖𝑗 𝐾 𝑘=1 ) 1 𝐾 (11) IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 477 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 The standard deviation 𝑝𝑤𝑐SD of individual judgments (Equation 10) is used to estimate weight variations based on judgment variations. 4.2 Group consensus Although it is always possible to calculate a group result using WGM-AIJ, it does not makes sense in all cases. For example, if we have two opposite judgments for two criteria (x and 1/x), an aggregation will result in equal weights for both criteria. In fact, there is no consensus, and equal weights may result in a deadlocked situation to solve a decision problem. Therefore, it is necessary to measure the consensus for the aggregated group result. We use Shannon entropy and its partitioning in two independent components (alpha and beta diversity) to derive the AHP consensus indicator. We analyze the priority distribution of criteria among different decision makers. We use the following relations (Goepel, 2013): Shannon γ-entropy 𝐻𝛾 = ∑ (−𝑤avgln 𝑤avg) 𝑛 𝑗=1 (11) with 𝑤avg = 1 𝑘 ∑ 𝑤𝑗𝑘 𝑘 𝑗=1 Shannon α-entropy 𝐻𝛼 = 1 𝑘 ∑ ∑ (−𝑤𝑗ln 𝑤𝑗 ) 𝑛 𝑗=1 𝑘 𝑖=1 (12) Shannon β-entropy 𝐻𝛽 = 𝐻𝛾 − 𝐻𝛼 (13) The similarity measure S (Equation 14) depends on the number of criteria, and we used a linear transformation to map it to a range from 0 to 1 (Equation 15). 𝑆 = exp(− 𝐻𝛽 ) = 1 𝐷β (14) 𝑆 ∗ = S−𝐷α min 𝐷γ max⁄ 1−𝐷α min 𝐷γ max⁄ (15) In general Dα min = 1 and Dγ max = n. In the Analytic Hierarchy Process (AHP) minimum alpha entropy 𝐻α min = ln(𝐷α min) is a function of the maximum scale value M (M = 9 for the fundamental AHP scale) and the number of criteria n. 𝐻𝛼 min = − ( 𝑀 𝑛+𝑀−1 ) 𝑙𝑛 [ 𝑀 𝑛+𝑀−1 ] − ( 𝑛−1 𝑛+𝑀−1 ) 𝑙𝑛 [( 1 𝑛+𝑀−1 )] (16) Maximum gamma entropy 𝐻γ max = ln (𝐷γ max) is 𝐻𝛾 max = ln (𝑛) (17) The correction factor cor is 𝑐𝑜𝑟 = 𝑒𝑥𝑝[𝐻𝛾 max − 𝐻𝛼 min] −1 = 𝐷α min 𝐷γ max⁄ = exp (𝐻𝛼 min) 𝑛 (18) IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 478 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 The AHP consensus indicator is based on the relative homogeneity S 𝑆 = 1 𝐷𝛽 − 1 𝐷𝛽 min (19) AHP Group Consensus Indicator 𝑆 ∗ = ( 1 exp (𝐻𝛽) − 1 exp (𝐻𝛾 max−𝐻𝛽 min) ) / (1 − 1 exp (𝐻𝛾 max−𝐻𝛽 min) ) (20) 𝑆 ∗ = 1 D𝛽 ⁄ − 𝑐𝑜𝑟 1− 𝑐𝑜𝑟 (21) The consensus of the complete hierarchy is calculated as weighted arithmetic mean of the consensus of all hierarchy nodes (categories); for alternatives, we take the consensus indicator of the consolidated alternative evaluation matrix weighted with the global priorities. The AHP consensus indicator (Equation 21) ranges from 0% (no consensus) to 100% (full consensus). The whole range is categorized into five categories according Table 2. Table 2 Interpretation of AHP consensus indicator S * S* Consensus ≤ 50% Very low 50% - 65% low 65% - 75% moderate 75% - 85% high ≥85% Very high 5. Weight uncertainties As shown by Goepel (2019), weight uncertainties due to rounding of the judgment to integers can exceed 10% and could affect the results of a decision. In order to get an estimation of uncertainties, we generate NVAR randomized variations of the original judgments with 𝑝𝑤𝑐𝑟𝑗 = (𝑎1, … 𝑎𝑖 , … 𝑎𝑛𝑝𝑐 ), (𝑥1 + ∆𝑥1, … 𝑥𝑖 + ∆𝑥𝑖 , … , 𝑥𝑛𝑝𝑐 + ∆𝑥𝑛𝑝𝑐) (22) With 𝑎1 = 0 for 𝑥𝑖 + ∆𝑥𝑖 ≥ 1, and 𝑎1 = 1 for 𝑥𝑖 + ∆𝑥𝑖 < 1, where ∆𝑋 are randomly uniform distributed values (-0.5, 0., +0.5). ∆𝑋~ 0.5 ∙ 𝑈(−1 , +1)/√𝐾 (23) IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 479 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 and K is the number of participants. For each variation, 𝑝𝑤𝑐𝑟𝑗 the eigenvector solution 𝑤𝑗 and consistency ratio CR is calculated. We then capture the maximum and minimum of the weights for all CR < 0.25. ∆𝑤(+) = (𝑤𝑚𝑎𝑥 − 𝑤) = (max[𝑤𝑗 = 𝐸𝑉(𝑝𝑤𝑐𝑟𝑗 )]𝑗=1 𝑁𝑉𝐴𝑅 − 𝑤) (24) ∆𝑤(−) = (𝑤 − 𝑤min) = (𝑤 − min[𝑤𝑗 = 𝐸𝑉(𝑝𝑤𝑐𝑟𝑗 )]𝑗=1 𝑁𝑉𝐴𝑅 ) (25) Figure 3 shows a typical example of weight uncertainties of a project with four criteria using the standard AHP 1 to 9 scale. Figure 3 Calculated weights for a project with four criteria showing weight uncertainties due to rounding Alternatively, users can also display uncertainties using the standard deviation of judgments 𝑝𝑤𝑐SD (Equation 10) if there is more than one participant. It gives, in addition to the AHP consensus, an idea about the weight variations based on participant’s judgment variations. 𝑝𝑤𝑐𝑟𝑗 = (𝑎1, … 𝑎𝑖 , … 𝑎𝑛𝑝𝑐 ), (𝑥1∆𝑥1, … 𝑥𝑖 ∆𝑥𝑖 , … , 𝑥𝑛𝑝𝑐∆𝑥𝑛𝑝𝑐 ) (26) ∆𝑋~𝑁(𝜇, 𝜎2) (27) ∆𝑥𝑖 = exp( 1 2 log 𝑝𝑤𝑐SD ∙ ∆𝑋) (28) 5.1 Overlap The program provides a list of criteria or alternatives overlapping within the estimated weight uncertainties. Overlapping criteria or alternatives should get the same ranking, as they cannot be differentiated within the estimated uncertainties. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 480 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 6. Alternative evaluation 6.1 Weighted sum and weighted product model In AHP, the preference Pi of alternative Ai is calculated using the weighted sum model (WSM): 𝑃𝑖 = ∑ 𝑎𝑖𝑗 𝑤𝑗 𝑛 𝑗=1 (29) with Wj the weight of criterion Cj, and aij the performance measure of alternative Ai with respect to criterion Cj. Performance values are normalized. ∑ 𝑎𝑖𝑗 𝑛 𝑗=1 = 1 (30) Users can also select the weighted product model (WPM), where alternatives are aggregated using the product instead of the sum 𝑃𝑖 = ∏ 𝑎𝑖𝑗 𝑤𝑗𝑛 𝑗=1 (31) Some of the first references to this method are due to Bridgman (1922) and Miller and Starr (1969). 7. Sensitivity analysis Sensitivity analysis is a fundamental concept in the effective use and implementation of quantitative decision models, whose purpose is to assess the stability of an optimal solution under changes in the parameters. Sensitivity analysis will answer two questions (Triantaphyllou, 1997):  Which is the most critical criterion?, and  Which is the most critical performance measure, changing the ranking between two alternatives? The most critical criterion is defined as the criterion Ck, with the smallest absolute (𝛿𝑘,𝑖,𝑗 abs ) or relative (𝛿𝑘,𝑖,𝑗 rel ) change of the current weight Wk by the amount of δkij changing the ranking between the alternatives Ai and Aj. The most critical measure of performance is defined as the minimum absolute or relative change of the current value of aij such that the current ranking between alternative Ai and Aj will change. The calculation is different for WSM and WPM. 7.1 Weighted sum model (WSM) To find the most critical criterion we calculate for each pair of alternatives Ai, Aj, with i = 1 to n and i < j 𝛿𝑖,𝑗,𝑘 abs (𝐴𝑖 , 𝐴𝑗 , 𝑊𝑘 ) = 𝑃𝑗−𝑃𝑖 𝑎𝑗𝑘−𝑎𝑖𝑘 (32) with |𝛿𝑖,𝑗,𝑘 abs (𝐴𝑖 , 𝐴𝑗 , 𝑊𝑘 )| < 𝑊𝑘 and 𝛿𝑖,𝑗,𝑘 rel (𝐴𝑖 , 𝐴𝑗 , 𝑊𝑘 ) = 𝛿𝑘,𝑖,𝑗 abs /𝑊𝑘 IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 481 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 To find the most critical measure of performance we calculate for all alternatives Ai and Ak with i ≠ k and each criterion 𝛿𝑖,𝑗,𝑘 abs (𝑊𝑗 , 𝑎𝑘𝑗 , 𝑎𝑖𝑗 ) = 𝑃𝑖−𝑃𝑘 𝑃𝑖−𝑃𝑘+𝑊𝑗(𝑎𝑘𝑗−𝑎𝑖𝑗+1) (33) with |𝛿𝑖,𝑗,𝑘 abs (𝑊𝑗 , 𝑎𝑘𝑗 , 𝑎𝑖𝑗 )| < 𝑊𝑗 and 𝛿𝑖,𝑗,𝑘 rel (𝑊𝑗 , 𝑎𝑘𝑗 , 𝑎𝑖𝑗 ) = 𝛿𝑘,𝑖,𝑗 abs /𝑎𝑖𝑗 (34) 7.2 Weighted product model (WPM) To find the most critical criterion for each pair of alternatives Ai, Ak, with i = 1 to n and i < k we calculate log (∏ (𝑎𝑖𝑦 𝑎𝑘𝑦⁄ ) 𝑊𝑦𝑛 𝑦=1 ) (35) and 𝛿(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) = log(∏ (𝑎𝑖𝑦 𝑎𝑘𝑦⁄ ) 𝑊𝑦𝑛 𝑦=1 ) log(𝑎𝑖𝑗 𝑎𝑘𝑗⁄ ) (36) with 𝛿abs(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) < 𝑊𝑗 Then we look at the smallest 𝛿rel(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) = 𝛿abs 𝑊𝑗 (37) To find the most critical measure of performance for the WPM we calculate for all alternatives Ai and Ak with i ≠ k and each criterion 𝛿rel(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) = (1 − √ 𝐴𝑘 𝐴𝑖 𝑊𝑗 ) (38) and look for the smallest 𝛿rel(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) < 1 (39) and 𝛿abs = 𝑎𝑖𝑗 𝛿 rel (40) 7.3 Program output for sensitivity analysis Depending on the results, the program will then display the following text output: a) The solution for the top alternative Ax is robust. b) The percent-top critical criterion is Cy, a change from y% by absolute dy% will change the ranking between alternatives Ai and Ak. c) The percent-any critical criterion is the same as above. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 482 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 d) The percent-any critical criterion is Cz, a change from z% by absolute dz% will change the ranking between alternatives Ai and Aj. e) The percent-any critical performance measure is for alternative Al under criterion Cz. A change from z% by absolute dz% will change the ranking between Ai and Aj. When exporting the results, the complete sensitivity tables (Equation 32 and 33 or Equations 36 and 40) will be exported. 8. Data export AHP-OS is intended for educational and research purposes, therefore it is important to provide an extensive possibility of data export for further analysis or presentation. The most common and universal format is the comma separated value (csv) text format. It can be easily imported into most spreadsheet programs, independent from the operation system. The software provides download functions for input data (hierarchy, decision matrices), local and global weights with estimated uncertainties, alternative tables and sensitivity analysis. 9. Software validation Coding and implementation of the complete AHP-OS software package resulted in approximately ten thousand lines of code; therefore, testing and validation became essential. With each new release, the author goes through a pre-defined checklist for unit, integration and system testing. In addition, in order to ensure that the user can rely on the accuracy of the program results, test cases were designed for black box testing. We compared program results with manually calculated results and results published in the literature. A simple test case with known results is the case where we judge one criterion x-times more important than all others. The weight for this criterion has to be (Goepel, 2019) 𝑤AHP = 𝑥 𝑥+𝑛−1 (41) For example, if all criteria are equal (x = 1), it follows that 𝑤AHP = 1/𝑛, and for four criteria, with one criterion nine times more important than all others, it follows 𝑤AHP = 9 12 𝑜𝑟 75%. In the next step we can apply different scales functions according table 1 by replacing x with the scaled value c. Group decision making can be validated by verifying that the elements of the consolidated group matrix show the geometric mean of the elements of the individual matrices. Testing the AHP consensus indicator is done by simulating k participants for a hierarchy with k leafs, and each participant judges one different leaf extreme more important than all others. The resulting consensus should show 0% for all scales. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 483 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 As a test case for weight uncertainties we use the derivate of Equation 41 d𝑤 d𝑥 = 𝑛−1 (𝑥+𝑛−1)2 (42) With n = 2, dx = 0.5 and x’ = x ± dx/2, x > 1 we get ∆𝑤(+) = 0.5 (𝑥+1.25)2 (43) ∆𝑤(−) = 0.5 (𝑥−0.75)2 (44) For example, for two criteria A and B with A three times more important than B, we get w = 75% with ∆𝑤(+) = 2.8% and ∆𝑤(−) = 3.6%. Alternative evaluation using the weighted sum or weighted product model can be verified exporting the results and using a spreadsheet program and its SUMPRODUCT function. As a test case for sensitivity analysis we use a spreadsheet program with exported data and subtract 𝛿abs from the original weight 𝑊𝑗 W′𝑘 = 𝑊𝑘 − 𝛿 abs (45) Using WSM Equation 29 or WPM Equation 31 for alternative aggregation the performance measure of alternative Ai should now equal that of alternative Aj. In a similar way we can calculate 𝑎′𝑖𝑗 = 𝑎𝑖𝑗 − 𝛿 abs (46) and alternative Ai should have the same performance measure as alternative Ak. For WSM we have to renormalize 𝑎𝑖𝑗 according Equation 30. 10. Experience and outlook AHP-OS was originally released in 2014, and features described in this paper were developed gradually over the last three years. With the currently implemented functionality, the software tool has reached a state where it covers most of the possible options for the classical Analytic Hierarchy Process. It allows for a wide range of parameter variations to analyze and study a specific project under different aspects. This and the possibility of transparent data export makes it an ideal tool for study, education and further research. Up to the date of writing this paper, more than 7000 users have registered for the software, and on average, there are at least 500 active users over a three-month period. AHP projects handled with AHP-OS cover a wide range of applications like healthcare, climate, risk assessment, supplier selection, hiring, IT, marketing, environment, transport, project management, manufacturing or quality assurance. Numerous projects are used to acquire group inputs; the number of participants goes up to 320 in a single project. Multi- language support is utilized for languages like Chinese, Korean, Russian, Hebrew, Greek, IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 484 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 Thai, Vietnamese or Arabic. Lecturers at universities make use of AHP-OS as a tool for their students when teaching multi-criteria decision-making methods. Overall, the number of software bugs has been kept reasonably low, more often user’s feedback and questions relate to the understanding of the AHP and have straightforward solutions. The main challenges in maintaining the software include updates of the underlying open source tools due to security reasons, or smaller updates requested by users. These updates always require extensive testing and validation because of the complexity of the software. A few extensions of AHP-OS are under consideration for the future. For example, a selection of different group aggregation techniques in addition to WGM-AIJ. As a major extension, a cluster analysis of group consensus is planned. The concept of Shannon entropy and its partitioning in alpha- and beta-entropy opens the possibility to identify clusters of sub-groups with high consensus within a group of decision makers. This could help to gain a deeper insight into group decision-making processes. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 485 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 REFERENCES Alonso, J. A., Lamata, T. (2006). Consistency in the Analytic Hierarchy Process: A new approach. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 14(4), 445−459. Doi: https://doi.org/10.1142/S0218488506004114 Bridgman, P.W. (1922). Dimensional Analysis. (Revised edition). New Haven, CT, U.S.A.: Yale University Press. Goepel, K. D. (2013). Implementing the Analytic Hierarchy Process as a standard method for Multi-Criteria Decision Making in corporate enterprises – A new AHP Excel template with multiple inputs. Proceedings of the International Symposium on the Analytic Hierarchy Process for Multicriteria Decision Making, Kuala Lumpur, Malaysia. Goepel, K. D. (2019). Comparison of judgment scales of the Analytical Hierarchy Process - A new approach. International Journal of Information Technology and Decision Making Vol.17(2019). DOI: 10.1142/S0219622019500044 Grošelj, P. Stirn, L. Z., Ayrilmis, N., Kuzman, M. K. (2015). Comparison of some aggregation techniques using group Analytic Hierarchy Process. Expert Systems with Applications, 42, 2198–2204. Doi: https://doi.org/10.1016/j.eswa.2014.09.060 Harker, P., Vargas, L. (1987). The theory of ratio scale estimation: Saaty's Analytic Hierarchy Process. Management Science, 33(11), 1383–1403. Doi: https://doi.org/10.1287/mnsc.33.11.1383 Ishizaka,A., Labib, A. (2009). Analytic Hierarchy Process and Expert Choice: Benefits and limitations. OR Insight, 22(4), 201–220. Doi: https://doi.org/10.1057/ori.2009.10 Ishizaka, A., Balkenborg, D., Kaplan, T. (2010). Influence of aggregation and measurement scale on ranking a compromise alternative in AHP. Journal of the Operational Research Society 62, 700–710. Doi: https://dx.doi.org/10.2139/ssrn.1527520 Larsen, R. (2013). Elementary linear algebra. Boston, MA: Cengage Learning. Lootsma, F. (2008). Conflict resolution via pairwise comparison of concessions. European Journal of Operational Research, 40,109–116. Doi: https://doi.org/10.1016/0377-2217(89)90278-6 Ma, D., & Zheng, X. (1991). 9/9–9/1 scale method of AHP. 2nd International Symposium on AHP, 1, 197–202. Miller, D.W., Starr, M.K. (1960). Executive decisions and operations research. Englewood Cliffs, NJ, U.S.A.: Prentice-Hall Inc. Ossadnik, W., Kaspar, R. (2013). Evaluation of AHP software from a management accounting perspective. Journal of Modelling in Management, 8(3), 305-319. Doi: https://doi.org/10.1108/JM2-01-2011-0007 IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 486 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 Saaty, T. L. (1980). The Analytic Hierarchy Process: Planning, priority setting, resource allocation. McGraw-Hill. Doi: https://doi.org/10.1016/0377-2217(82)90022-4 Saaty, T.L. (2003). Decision-making with the AHP: Why is the principal eigenvector necessary. European Journal of Operational Research, 145, 85–91. Doi: https://doi.org/10.1016/S0377-2217(02)00227-8 Salo, A., Hämäläinen, R. (1997). On the measurement of preferences in the Analytic Hierarchy Process. Journal of Multi-Criteria Decision Analysis, 6, 309–319. Doi: https://doi.org/10.1002/(SICI)1099-1360(199711)6:6%3C309::AID- MCDA163%3E3.0.CO;2-2 Siraj, S., Mikhailov, L., Keane, J. A., (2015). An interactive decision support tool to estimate priorities from pairwise comparison judgments (PriEsT). International Transactions in Operational Research, 22(2), 217–235. Doi: https://doi.org/10.1111/itor.12054 Triantaphyllou, E., Sánchez, A. (1997). Sensitivity analysis approach for some deterministic Multi-Criteria Decision Making Methods. Decision Sciences, 28(1), 151- 194. Doi: https://doi.org/10.1111/j.1540-5915.1997.tb01306.x Wen-Hsiang Wu, Chang-tzu Chiang, Chin-tsai Lin (2008). Comparing the aggregation methods in the Analytic Hierarchy Process when uniform distribution. WSEAS Transactions On Business And Economics, 5(3), 82 – 87. IJAHP Article: Goepel/ Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS) International Journal of the Analytic Hierarchy Process 487 Vol. 10 Issue 3 2018 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 APPENDIX 1. Open source software packages used for AHP-OS PHP, Version 5.6.31, http://php.net/ MariaDB, MariaDB Foundation, Version 5.5.54, https://mariadb.org/ SQLite, Version 3.8.10.2, https://sqlite.org/index.html jquery Java script library, https://jquery.com/ PHPGraphLib Graphing Library, Elliot Brueggeman, http://www.ebrueggeman.com/phpgraphlib PHPMailer, the classic email-sending library for PHP, https://github.com/PHPMailer/PHPMailer 2. Other AHP software packages 123AHP, http://123ahp.com/ AHP-OS, https://bpmsg.com/academic/ahp.php MakeItRational, http://makeitrational.com ChoiceResults, https://www.filebuzz.com/fileinfo/91865/ChoiceResults.html Decisor, https://github.com/czekster/Decisor/ HIPRE 3+, http://sal.aalto.fi/en/resources/downloadables/hipre3 SuperDecisions, https://superdecisions.com/ Expert Choice, https://expertchoice.com/ Decision Lens, https://decisionlens.com CDP, Criterium® DecisionPlus®, http://www.infoharvest.com/ihroot/index.asp RightChoiceDSS, https://www.updatestar.com/publisher/tgkconsulting-2369 PriEsT, AHP Priority Estimation Tool, https://sourceforge.net/projects/priority/